Measuring What Matters: Objective Metrics for Image Generation Assessment
β’
9
AI & ML interests
Efficient machine learning for any model and hardware: pruning, quantization, compilation, and more.
Recent Activity
View all activity
Articles
johnrachwanprunaΒ
updated
a
model
1 day ago
davidberenstein1957Β
published
a
dataset
12 days ago
davidberenstein1957Β
updated
a
Space
18 days ago
johnrachwanprunaΒ
updated
2
models
about 1 month ago
davidberenstein1957Β
posted
an
update
3 months ago
Post
472
Announcing RealPerformance, a dataset of functional issues of language models that mirrors failure patterns identified through rigorous testing in real LLM agents
https://huggingface.co/blog/davidberenstein1957/realperformance-llm-business-compliance
https://huggingface.co/blog/davidberenstein1957/realperformance-llm-business-compliance
davidberenstein1957Β
posted
an
update
4 months ago
Post
376
π¨ LLMs recognise bias but also reproduce harmful stereotypes: an analysis of bias in leading LLMs
I've written a new entry in our series on the Giskard, BPIFrance and Google Deepmind Phare benchmark(phare.giskard.ai).
This time it covers bias: https://huggingface.co/blog/davidberenstein1957/llms-recognise-bias-but-also-produce-stereotypes
Previous entry on hallucinations: https://huggingface.co/blog/davidberenstein1957/phare-analysis-of-hallucination-in-leading-llms
I've written a new entry in our series on the Giskard, BPIFrance and Google Deepmind Phare benchmark(phare.giskard.ai).
This time it covers bias: https://huggingface.co/blog/davidberenstein1957/llms-recognise-bias-but-also-produce-stereotypes
Previous entry on hallucinations: https://huggingface.co/blog/davidberenstein1957/phare-analysis-of-hallucination-in-leading-llms
- 1 reply
davidberenstein1957Β
posted
an
update
5 months ago
Post
1746
I created a collection of FLUX.1 models but 4x faster
PrunaAI/flux1-but-4x-faster-66c0b7340836dd7a55e9c0ea
davidberenstein1957Β
posted
an
update
6 months ago
Post
1075
Good answers are not necessarily factual answers: an analysis of hallucination in leading LLMs
https://huggingface.co/blog/davidberenstein1957/phare-analysis-of-hallucination-in-leading-llms
https://huggingface.co/blog/davidberenstein1957/phare-analysis-of-hallucination-in-leading-llms
Post
3186
How to learn about efficient AI? - Happy to announce the Awesome AI Efficiency repo that gathers a curated list of 100+ materials to understand the challenges and solutions in making AI faster, smaller, cheaper, greener.
π It is designed for a **large audience** including beginners, decision-makers, engineers, and researchers.
π It contains **diverse materials** with newspaper articles, blogs, tools, tech reports, research papers, books, and lectures.
This is an ongoing project. Do not hesitate to share your feedback/suggestions and star the repo! π
https://github.com/PrunaAI/awesome-ai-efficiency
π It is designed for a **large audience** including beginners, decision-makers, engineers, and researchers.
π It contains **diverse materials** with newspaper articles, blogs, tools, tech reports, research papers, books, and lectures.
This is an ongoing project. Do not hesitate to share your feedback/suggestions and star the repo! π
https://github.com/PrunaAI/awesome-ai-efficiency
- 2 replies
davidberenstein1957Β
posted
an
update
6 months ago
Post
2254
π₯ Announcing FLUX-Juiced: The Fastest Image Generation Endpoint (2.6x faster)!
Optimisations are widely applied and can reduce inference time, but their impact on quality often remains unclear, so we decided to challenge the status quo and create our own optimised version of FLUX.1[dev] called FLUX-juiced.
Blog: https://huggingface.co/blog/PrunaAI/flux-fastest-image-generation-endpoint
Optimisations are widely applied and can reduce inference time, but their impact on quality often remains unclear, so we decided to challenge the status quo and create our own optimised version of FLUX.1[dev] called FLUX-juiced.
Blog: https://huggingface.co/blog/PrunaAI/flux-fastest-image-generation-endpoint
davidberenstein1957Β
posted
an
update
6 months ago
Post
1732
π§βπ« I wrote a brief blogpost to give An Introduction to AI Model Optimization Techniques!
URL: https://huggingface.co/blog/PrunaAI/introduction-to-ai-model-optimization-techniques
URL: https://huggingface.co/blog/PrunaAI/introduction-to-ai-model-optimization-techniques
davidberenstein1957Β
posted
an
update
6 months ago
Post
1402
RealHarm: A Collection of Real-World Language Model Application Failure
I'm David from Giskard, and we work on securing your Agents.
Today, we are launching RealHarm: a dataset of real-world problematic interactions with AI agents, drawn from publicly reported incidents.
Check out the dataset and paper: https://realharm.giskard.ai/
I'm David from Giskard, and we work on securing your Agents.
Today, we are launching RealHarm: a dataset of real-world problematic interactions with AI agents, drawn from publicly reported incidents.
Check out the dataset and paper: https://realharm.giskard.ai/
davidberenstein1957Β
posted
an
update
7 months ago
Post
2108
π¨ New Bonus Unit: Tracing & Evaluating Your Agent! π¨
Learn how to transform your agent from a simple demo into a robust, reliable product ready for real users.
UNIT: https://huggingface.co/learn/agents-course/bonus-unit2/introduction
In this unit, you'll learn:
- Offline Evaluation β Benchmark and iterate your agent using datasets.
- Online Evaluation β Continuously track key metrics such as latency, costs, and user feedback.
Happy testing and improving!
Thanks Langfuse team!
Learn how to transform your agent from a simple demo into a robust, reliable product ready for real users.
UNIT: https://huggingface.co/learn/agents-course/bonus-unit2/introduction
In this unit, you'll learn:
- Offline Evaluation β Benchmark and iterate your agent using datasets.
- Online Evaluation β Continuously track key metrics such as latency, costs, and user feedback.
Happy testing and improving!
Thanks Langfuse team!
Post
3111
We open-sourced the
- Github repo: https://github.com/PrunaAI/pruna
- Documentation: https://docs.pruna.ai/en/stable/index.html
With open-sourcing, people can now inspect and contribute to the open code. Beyond the code, we provide detailed readme, tutorials, benchmarks, and documentation to make transparent compression, evaluation, and saving/loading/serving of AI models.
Happy to share it with you and always interested in collecting your feedback :)
pruna package that can be easily installed with pip install pruna :) It allows to easily ccompress and evaluate AI models including transformers and diffusers.- Github repo: https://github.com/PrunaAI/pruna
- Documentation: https://docs.pruna.ai/en/stable/index.html
With open-sourcing, people can now inspect and contribute to the open code. Beyond the code, we provide detailed readme, tutorials, benchmarks, and documentation to make transparent compression, evaluation, and saving/loading/serving of AI models.
Happy to share it with you and always interested in collecting your feedback :)
- 2 replies
davidberenstein1957Β
posted
an
update
8 months ago
Post
2432
π₯ Text2SQL, explore and share any data analysis!
π€ Hugging Face - Dataset Studio is an amazing new feature.
π Start yourself: fka/awesome-chatgpt-prompts
πΊ YouTube: https://youtu.be/5LUZq7MHolA?feature=shared
π€ Hugging Face - Dataset Studio is an amazing new feature.
π Start yourself: fka/awesome-chatgpt-prompts
πΊ YouTube: https://youtu.be/5LUZq7MHolA?feature=shared
davidberenstein1957Β
posted
an
update
8 months ago
Post
4262
π₯ Epic Agent Framework Showdown! Available today!
π΅ In the blue corner, the versatile challenger with a proven track record of knowledge retrieval: LlamaIndex!
π In the red corner, the defender, weighing in with lightweight efficiency: Hugging Face smolagents!
π URL:
agents-course
We just published the LlamaIndex unit for the agents course, and it is set to offer a great contrast between the smolagents unit by looking at
- What makes llama-index stand-out
- How the LlamaHub is used for integrations
- Creating QueryEngine components
- Using agents and tools
- Agentic and multi-agent workflows
The team has been working flat-out on this for a few weeks. Supported by Logan Markewich and Laurie Voss over at LlamaIndex.
Who won? You decide!
π΅ In the blue corner, the versatile challenger with a proven track record of knowledge retrieval: LlamaIndex!
π In the red corner, the defender, weighing in with lightweight efficiency: Hugging Face smolagents!
π URL:
agents-course
We just published the LlamaIndex unit for the agents course, and it is set to offer a great contrast between the smolagents unit by looking at
- What makes llama-index stand-out
- How the LlamaHub is used for integrations
- Creating QueryEngine components
- Using agents and tools
- Agentic and multi-agent workflows
The team has been working flat-out on this for a few weeks. Supported by Logan Markewich and Laurie Voss over at LlamaIndex.
Who won? You decide!
davidberenstein1957Β
posted
an
update
8 months ago
Post
3053
π«Έ New release to push vector search to the Hub with vicinity and work with any serialisable objects.
π§βπ« KNN, HNSW, USEARCH, ANNOY, PYNNDESCENT, FAISS, and VOYAGER.
π Example Repo: minishlab/my-vicinity-repo
π§βπ« KNN, HNSW, USEARCH, ANNOY, PYNNDESCENT, FAISS, and VOYAGER.
π Example Repo: minishlab/my-vicinity-repo
davidberenstein1957Β
posted
an
update
9 months ago
Post
3324
π Find banger tools for your smolagents!
I created the Tools gallery, which makes tools specifically developed by/for smolagents searchable and visible. This will help with:
- inspiration
- best practices
- finding cool tools
Space: davidberenstein1957/smolagents-and-tools
I created the Tools gallery, which makes tools specifically developed by/for smolagents searchable and visible. This will help with:
- inspiration
- best practices
- finding cool tools
Space: davidberenstein1957/smolagents-and-tools
- 1 reply
davidberenstein1957Β
posted
an
update
9 months ago
Post
2490
Fine-tune Deepseek-R1 with a Synthetic Reasoning Dataset
Blog: https://huggingface.co/blog/sdiazlor/fine-tune-deepseek-with-a-synthetic-reasoning-data
Blog: https://huggingface.co/blog/sdiazlor/fine-tune-deepseek-with-a-synthetic-reasoning-data