Omniroute

[00:00] This is the repository for something called OmniRoute. And OmniRoute is a gateway for like a gazillion AI providers. Here’s their website, never stop coding, blah, blah, blah. We’ll jump to start and you install it with NPM. There’s a couple of different ways to install it. You could run it as a Docker file or a Docker container if you like. But it’s npm install-g omniroute, and then you run omniroute.

[00:29] Now I’ve already got it installed. I’m going to run omniroute and it is up and running. I’m going to jump over here to localhost:20128. And you get this groovy little dashboard. So steps one through four, you want to go in, you want to create an API key. Now this is an API key for OmniRoute on your own machine. It’s not for some external service. I’ve already got one set up here, but say create OmniRoute API key. You give it a name, you hit the create key, and you get a key, which you can then use. And we’ll get into that in just a second. Can jump back to home.

[01:06] The next thing you want to do, step two, is jump into providers. Now I’ve got a bunch configured here, but I just want to point out, there are a shit ton of providers. A bunch I’ve never heard of. Some were easy to sign up. Some weren’t, but for the most part, I just, I did 15 of these free tier ones. And you go in, you click on the guy, you go to the website, you sign up, you do whatever you got to do. And then you hit add and you plug in your API key. And again, this is all local. This is all running on my own machine or your own machine. So once you’ve got a bunch of those set up, and I will point out, I actually set up OMLX as a provider. So if all else fails, this will fall back to my local inference engine. And like I said, I don’t even know who a lot of these companies are. I just like free shit. So that’s what I’m doing.

[02:03] Okay. Now, the next thing I’m going to show you really quick is this combos thing. So combos is really cool. And you can create your own down here. But what they have is these built-in ones. And this is very similar to if you’ve ever used, I think OpenRouter has something like this, where you just say, like, give me the cheapest model or the fastest model or whatever. And it just figures out what it’s going to do for you. So they have some weighted collections here. Best coding, best reasoning, best fast, best coding fast, pro coding, you know, on and on and on. And so that’s really handy. And we can use these in our configuration of our client as basically a stand-in for the model name. So instead of saying, well, I want Quen 3.6 this and GLM 5.2 that, we’re going to say, give me the best coding ones or whatever is available. And it will cycle through our providers trying to find the best match.

[03:05] So what I’m going to do is jump into my open code configuration here. So this is, if you’re on a Mac, it’s home.config open code, then opencode.json. And I’ve got these providers set up, like MLX, DSPARC, and OMLX. And what I’m going to do is I’m just going to add another provider. Now, this is going to be OmniRoute. The configuration is just like the other ones, NPM, AI SDK. I give it a name, some options. So the route is, and it has both OpenAI-compatible and Anthropic-compatible routes for you. I’m just doing OpenAI-compatible here, the API key for OmniRoute. So OmniRoute has all the API keys for the providers. And then this is my API key for OmniRoute. It’s not going to do you any good. It’s only here on my machine. And then for models, I’m setting up AutoProCoding, AutoBestCodingFast, and AutoBestFree. So I’m just trying these out. We’re going to see how it goes. So I’m going to save that.

[04:11] And then what I’m going to do is I have this little project called SaveFod. That’s like how I use AI to manage my YouTube channel. And when I go into videos right now, the default sort is on view. So I’m just going to tell it to make the default sort date, and we’re going to see how that goes. So I’m going to load this guy up. We’re going to jump into the project, say OpenCode. I’m going to put this over there. And we are going to jump into our provider topology. Now, what this is going to do is show me which services it is accessing. Now, I don’t have OMLX running at this very moment because recording and OBS and all that stuff at the same time, it gets a little too crazy. So I’ve got all the rest of them enabled. But that one, if it hits that, that is definitely going to fail.

[05:07] So in here, in OpenCode, I’m going to switch my model to auto best coding fast. This is a really simple task. I don’t think I need like something really extreme. And I’m going to drop this prompt in. Okay. So I’m saying the table.js is defaulting to a sort on the views column, make it sort by default on the date column. And again, I’m not really picking a model here. I’m just saying go, just go get the best coding fast model, I guess. So I’m going to run that. And we’re going to see how this goes. So we can see it is hitting this Agnes AI. I think it’s going to get it one and done here. But I have run some longer running processes where it was like NVIDIA for a while. And then that got a little slow or wasn’t a good connection. So it jumped over to Agnes or Mistral or something like that. So this little chart here will jump around on a long running task and start picking different providers. So this says it’s done.

[06:11] I’m going to jump over to my little app here, go to videos and sure enough, it’s now being sorted by date. Yeah. So this is really cool. I’m going to keep exploring it. There is a ton of other stuff in here. Let me just bring this up. There’s let me zoom it out a little bit. There are. So when we get into this section, so all I’ve really looked at here is the API keys and the providers. You can set up your own end points. I showed you combos, but there’s also a whole bunch of optimizations you can put in place. So if I come down here to compression studio. No, that’s not it. Ah, compression settings. So I can set up prompt compression. You can enable headroom, caveman, RTK. There’s a really interesting one called OmniGlyph I’m going to be taking a look at, which is from the creator of OmniRoute. And if I’m correct on that, it actually converts your prompt to an image. And I don’t know how the fuck that works. That’s kind of awesome. But yeah, there’s a lot more to go in here or into here. I highly recommend you check this out. It’s a bit of setup, you know, in terms of creating all these providers and connecting to them and figuring out that some of them don’t work at all. But if you like trying out free shit, here’s a really easy way to do it. So I highly suggest you check out OmniRoute.