Nativ

[00:00] So the creator of mlx-vlm and also mlx-audio, he goes by Blaizzy, his name is Prince Canuma, just created a new application called Native, which is a MLX, I’m going to call it a studio, because it does more than just a regular inference engine. Here’s the website, there’s a bunch of good stuff there. We’re going to jump into it, we’ll do a little bit of comparison, but mostly I want to look at some of the different and interesting things it does that sets it aside from other inference engines.

[00:38] So right off the bat, it has a basic chat interface, I can come in and select a model, I can select its reasoning level, and we can chat with the model, all pretty straightforward. Now, if I jump over to this dev section, we can see all of the open AI compatible endpoints, as well as anthropic compatible endpoints. And if I scroll down here, we can see in this terminal window, it’s actually running mlx-vlm server.

[01:03] So I’ve done some comparisons in the past between different MLX inference engines. And of course, this turned out exactly as I expected, virtually identical to the standalone mlx-vlm server decoding and pre-fill stats. So no change there. Now, if we come to integrations, just like a lot of these guys, we have integrations where if I want to run PI against this or any other code against it, it is running a server.

[01:31] So I have access to those endpoints, I can click on open code, it’s going to give me some commands, I can choose which folder I wanted to open up in, and then I can load up open code running against the native mlx-vlm server. All right, so we go to models here. And now this is where it starts to get a little bit interesting for us. I’ve got it looking at all the different folders where I have models and it’s loaded those up. Of course, they do need to be MLX models. I can go over to discover and it’s going to use hugging face just like any other.

[02:11] But since this is a VLM engine, we can do things like image generation. So if I come over here and search for this little tiny bonsai image ternary 4 billion MLX 2-bit model, I can download that. Now I can jump over to chat. We’ll start a new chat, switch to images, and it’s already selected the one model I have. And I’ll just run this prompt that I kind of like to run just because I don’t know, I just do. And we get our image back, I can download it from there. And it’ll also be stored in this artifact section.

[02:50] So in this artifact section, it’s going to save any documents or images or anything that it makes. And I can blow that up and do whatever I want with it. So another thing that’s really nice about native is if I go over here to extensions and then tools, almost everything is opt in. So I had to turn all these on. And if I go to web read, I’ve already set it up with Tavoli, but you can choose however you want to set that up. You can also add custom tools, but there’s a whole bunch built in here that you can go ahead and turn on and configure as you see fit.

[03:28] It also comes with these kits, which are like preset little combinations of skills and tools that you can enable to do certain tasks. This one right here is the research kit. I do have that enabled. Now, another cool thing is this scheduler right here where we can create automations. So I don’t have any right now. I’m going to come in and say new scheduled task. So I’m going to call this guy fetch RSS. I’m going to tell it to fetch this RSS feed. I’m going to pick my model.

[03:54] I’m going to pick my tools and I’m just going to use research since that contains a whole bunch of other tools. I can set my time. Try to set that for one minute from now. And then I can just create it. And you’ll notice as soon as I created it, it created a session over here. So it treats these scheduled tasks like ongoing chat sessions. So we’ll give that a minute. And we can see it kicked off now at 5:09 PM.

[04:21] And I’ve got it scheduled right now to run every day at 5:09 PM. But obviously this could be doing all sorts of different things. It doesn’t have to be fetching anything. It could be running processes on my machine, something I need done every Tuesday and Thursday, whatever. Cool. So that wrapped up. It took about a minute. And if I jump over here, I’ve got my summary. It went through its whole process as though I was talking with it and it did exactly what I asked it to do.

[04:52] And again, since it’s a VLM engine, we can come in here and do something like say, take a screenshot. I could grab this section here. They extract the text from this image and it gives me back the text from the image. So that’s a quick look at native go follow everything that Prince makes mlx-vlm, mlx-audio and native. The guy’s doing terrific work. Be sure to check it out. I’m Joe Medalone. I make short practical videos that respect your intelligence and your time. If that’s your style, you’ll like this channel.