AI & ML interests

None defined yet.

Recent Activity

evijitย 
posted an update 28 days ago
view post
Post
2502
AI for Scientific Discovery Won't Work Without Fixing How We Collaborate.

My co-author @cgeorgiaw and I just published a paper challenging a core assumption: that the main barriers to AI in science are technical. They're not. They're social.

Key findings:

๐Ÿšจ The "AI Scientist" myth delays progress: Waiting for AGI devalues human expertise and obscures science's real purpose: cultivating understanding, not just outputs.
๐Ÿ“Š Wrong incentives: Datasets have 100x longer impact than models, yet data curation is undervalued.
โš ๏ธ Broken collaboration: Domain scientists want understanding. ML researchers optimize performance. Without shared language, projects fail.
๐Ÿ” Fragmentation costs years: Harmonizing just 9 cancer files took 329 hours.

Why this matters: Upstream bottlenecks like efficient PDE solvers could accelerate discovery across multiple sciences. CASP mobilized a community around protein structure, enabling AlphaFold. We need this for dozens of challenges.

Thus, we're launching Hugging Science! A global community addressing these barriers through collaborative challenges, open toolkits, education, and community-owned infrastructure. Please find all the links below!

Paper: AI for Scientific Discovery is a Social Problem (2509.06580)
Join: hugging-science
Discord: https://discord.com/invite/VYkdEVjJ5J
evijitย 
posted an update 4 months ago
view post
Post
398
New blog post alert! "What is the Hugging Face Community Building?", with @yjernite and @irenesolaiman

What 1.8 Million Models Reveal About Open Source Innovation: Our latest deep dive into the Hugging Face Hub reveals patterns that challenge conventional AI narratives:

๐Ÿ”— Models become platforms for innovation Qwen, Llama, and Gemma models have spawned entire ecosystems of specialized variants. Looking at derivative works shows community adoption better than any single metric.

๐Ÿ“Š Datasets reveal the foundation layer โ†’ Most downloaded datasets are evaluation benchmarks (MMLU, Squad, GLUE) โ†’ Universities and research institutions dominate foundational data โ†’ Domain-specific datasets thrive across finance, healthcare, robotics, and science โ†’ Open actors provide the datasets that power most AI development

๐Ÿ›๏ธ Research institutions lead the charge: AI2 (Allen Institute) emerges as one of the most active contributors, alongside significant activity from IBM, NVIDIA, and international organizations. The open source ecosystem spans far beyond Big Tech.

๐Ÿ” Interactive exploration tools: We've built several tools to help you discover patterns!

ModelVerse Explorer - organizational contributions
DataVerse Explorer - dataset patterns
Organization HeatMap - activity over time
Base Model Explorer - model family trees
Semantic Search - find models by capability

๐Ÿ“š Academic research is thriving: Researchers are already producing valuable insights, including recent work at FAccT 2025: "The Brief and Wondrous Life of Open Models." We've also made hub datasets, weekly snapshots, and other data available for your own analysis.

The bottom line: AI development is far more distributed, diverse, and collaborative than popular narratives suggest. Real innovation happens through community collaboration across specialized domains.

Read: https://huggingface.co/blog/evijit/hf-hub-ecosystem-overview
evijitย 
posted an update 5 months ago
clefourrierย 
posted an update 6 months ago
view post
Post
1753
Always surprised that so few people actually read the FineTasks blog, on
โœจhow to select training evals with the highest signalโœจ

If you're serious about training models without wasting compute on shitty runs, you absolutely should read it!!

An high signal eval actually tells you precisely, during training, how wel & what your model is learning, allowing you to discard the bad runs/bad samplings/...!

The blog covers in depth prompt choice, metrics, dataset, across languages/capabilities, and my fave section is "which properties should evals have"๐Ÿ‘Œ
(to know on your use case how to select the best evals for you)

Blog: HuggingFaceFW/blogpost-fine-tasks
  • 2 replies
ยท
clefourrierย 
posted an update 8 months ago
view post
Post
2639
Gemma3 family is out! Reading the tech report, and this section was really interesting to me from a methods/scientific fairness pov.

Instead of doing over-hyped comparisons, they clearly state that **results are reported in a setup which is advantageous to their models**.
(Which everybody does, but people usually don't say)

For a tech report, it makes a lot of sense to report model performance when used optimally!
On leaderboards on the other hand, comparison will be apples to apples, but in a potentially unoptimal way for a given model family (like some user interact sub-optimally with models)

Also contains a cool section (6) on training data memorization rate too! Important to see if your model will output the training data it has seen as such: always an issue for privacy/copyright/... but also very much for evaluation!

Because if your model knows its evals by heart, you're not testing for generalization.
clefourrierย 
posted an update over 1 year ago
view post
Post
6164
In a basic chatbots, errors are annoyances. In medical LLMs, errors can have life-threatening consequences ๐Ÿฉธ

It's therefore vital to benchmark/follow advances in medical LLMs before even thinking about deployment.

This is why a small research team introduced a medical LLM leaderboard, to get reproducible and comparable results between LLMs, and allow everyone to follow advances in the field.

openlifescienceai/open_medical_llm_leaderboard

Congrats to @aaditya and @pminervini !
Learn more in the blog: https://huggingface.co/blog/leaderboard-medicalllm
clefourrierย 
posted an update over 1 year ago
view post
Post
4795
Contamination free code evaluations with LiveCodeBench! ๐Ÿ–ฅ๏ธ

LiveCodeBench is a new leaderboard, which contains:
- complete code evaluations (on code generation, self repair, code execution, tests)
- my favorite feature: problem selection by publication date ๐Ÿ“…

This feature means that you can get model scores averaged only on new problems out of the training data. This means... contamination free code evals! ๐Ÿš€

Check it out!

Blog: https://huggingface.co/blog/leaderboard-livecodebench
Leaderboard: livecodebench/leaderboard

Congrats to @StringChaos @minimario @xu3kev @kingh0730 and @FanjiaYan for the super cool leaderboard!
clefourrierย 
posted an update over 1 year ago
view post
Post
2275
๐Ÿ†• Evaluate your RL agents - who's best at Atari?๐Ÿ†

The new RL leaderboard evaluates agents in 87 possible environments (from Atari ๐ŸŽฎ to motion control simulations๐Ÿšถand more)!

When you submit your model, it's run and evaluated in real time - and the leaderboard displays small videos of the best model's run, which is super fun to watch! โœจ

Kudos to @qgallouedec for creating and maintaining the leaderboard!
Let's find out which agent is the best at games! ๐Ÿš€

open-rl-leaderboard/leaderboard