I've been working on some new features for Anime Nano in the last few weeks, just to have something to work on. There's a feature that I disabled that I had set up in the early days of the site which would detect if a blog post was about an anime series, and if it had an episode number. So if someone was blogging about Naruto Episode 10, I could then show all the blog posts about that episode (or just about Naruto).

It originally worked by just checking a new blog entry against a giant list of regular expressions that I hand wrote for each series (usually they were just an exact match though sometimes they had to be a bit more complicated). It worked pretty well except that it was a pain to add new series for every season and after a while I just wasn't interested in managing it anymore.

So when I updated Anime Nano to run on Cloudflare last year, I only kept the bare minimum feature set. Since I have AI as my intern now, I've been having it whip up features for things that no one will use. And so I set it on a quest to automate the Anime Nano Entry Series Tagger!

An Actual Good Problem For AI

Whenever I think of how to make AI do something, I first think of whether AI would actually be a good fit for the thing I want to do. Because, let's face it, most stuff that I think of can just be done with a script. After a few years, I still hardly have any actual AI workflows where I trust AI to do a better job than a predefined script. And even if AI could do a better job, it's usually overkill and not necessary.

But for this particular problem, AI actually seems like a great fit! Let me give you an example. Here's a random post about an anime from this year:

As a human, if you wanted to figure out what series this post was about, you'd start by identifying what looks the most like an anime series title, in this case, "Sound! Euphonium". Then you might Google it to make sure that's an actual series. Then I guess you'd add an entry in the database or something, I dunno.

So that's exactly what I do to make AI auto-classify posts for me. I first run the title and snippet through an LLM that does named entity recognition on it to find the most likely anime title. There's actual models that are trained to do this but I just run this step on a real LLM because why not?

Then I make an API call to the MyAnimeList database and fetch a few potential matches. Finally, I make another LLM call that chooses a series from the list of responses, or just classifies the entry as not a match if it's not an episodic anime post.

The reason I do this in two steps is because I want an actual reference to a real anime series (not hallucinated by AI) and it helps with deduplicating since each MAL series has a unique ID.

The Scale of the Problem

If you haven't noticed by the design of the site, Anime Nano is 20 years old this year! I'm actually really proud of the fact that I've been able to keep this website up for 20 years, and I'm hella impressed that some people actually still publish anime blogs regularly!

There are like 430k posts, which means almost a million LLM calls that I'd need to make to classify all of these entries. I can actually hardly believe that people used to blog that hard! Anyway, I'm not one to actually pay to have clankers do meaningless work, so I set out to see if I could get all of these entries classified for (essentially) free.

So the rest of this post is gonna be about me hustling any AI labs that I can for that sweet free inference!

Sweet, free inference

I've been using OpenRouter for a lot of random, non-privacy-sensitive stuff lately and it's been pretty great. All I had to do was put $10 worth of credits on my account, and I get access to the free tier that gives me 1000 requests per day on their free tier models. One weird thing about OpenRouter is they rate limit on requests and not tokens, so it doesn't matter if you send "hi" or a huge novel, it still counts as a request. I was using this to prototype the feature, but I realized that doing 500 entries a day would take more than two years to complete the backfill process!

I saw a post on Hacker News recently about the hosting provider Hetzner, which is running an "experiment" to provide inference services. They're only offering one model, which happens to be my favorite one to run locally, Qwen3.6-35B-A3B! I signed up for a Hetzner account and added that to my list of models. As of this writing, their experimental free tier gives you this amount of usage:

Timeframe	Input Tokens	Output Tokens
60s	3M	60k
24h	500M	5M

I haven't done the math but I'm sure Hetzner buys me a few more entries processed per day!

Next, I signed up for NVIDIA NIM which also gives you some free usage every day. I didn't look into this one too much, though they have some really nice models like GLM-5.2 which is apparently gonna kill the US AI industry! I have it set to run a Nemotron model. Since the model is NVIDIA I figure it'll work pretty well.

I also signed up for Groq (not the xAI one) that provides super fast inference on custom hardware. I'm running a few different models from them, including some Llama and Qwen models.

Finally, I dug up my old Mistral API keys (okay actually I just made a new one) and started using their inference API on one of their small models. This was actually the best inference provider in terms of both latency and throughput. I'm not even sure if I've been rate limited on it at all, yet!

Of course, using all of these free providers means that they can probably retain and train on all of the stuff I'm sending them. But I'm just sending them snippets of weeaboo anime posts anyway, so joke's on them!

So with all of these free providers, I was classifying around 12 entries per minute. It would take around 24 days to complete, which is not too bad. But could I do better?

Enter Kaggle

I remembered that Kaggle has Jupyter Notebooks that you can run for 30 hours every week for free. I use them for things like training voice cloning models or making funny pictures (or videos) of myself doing stupid stuff. They provide either a P100 or dual T4s as GPU options. I got AI to set up a Kaggle notebook that served an inference server running the two T4s in parallel. I tried a bunch of models but settled on Qwen2.5-3B-Instruct since it easily fits and runs pretty quickly. I was a bit concerned that it was too old since 2 years is an eternity in LLM time. But it's definitely smart enough to do the simple tasks that I'm asking of it.

With vLLM I'm able to process around 168 entries per minute, which means I could finish backfilling Anime Nano in a few days. At least, I would if I didn't have that 30-hour limit for this week.

Why I Did It

I completely recognize that it's utterly pointless to classify 20-year-old blog posts on Anime Nano that are probably 404ing anyway. But as I'm currently unemployed, I gotta find something to do with my time! Ever since I started writing code with AI, I've thought that this would be a fun project to set up, and it really has been! I've been waking up in the morning to check in on the progress, and debugging when I don't get the results I expect.

For me, it really isn't about classifying anime series. It's more about the process of experimenting with AI on a non-trivial system. I've already learned a whole lot more about things like scaling inference, and the power of small open source models when solving fairly simple problems. If I was really serious about efficiency, I'd probably also experiment with some of the NER models instead of doing that step with an LLM. Once the backfill finishes, I'll probably just go with a larger, more capable model to handle any new posts on Anime Nano to ensure super high accuracy. Then I'll probably have to find some other high effort, low impact feature to add!