How to install a 100% PRIVATE local AI chatbot

00:00:00: And today I'm gonna show you how you can install a full blown AI chatbot on your own computer.

00:00:08: Why does that matter?

00:00:10: Well, because I did a survey on my LinkedIn and almost half of my patent attorney colleagues prefer local AI for patent drafting.

00:00:19: So I'm gonna install everything and we're gonna go through the whole setup process together on my computer right now.

00:00:24: The first thing you wanna do is you wanna open up a browser and go to olama.com.

00:00:31: You click on download and then you pick your version for your operating system.

00:00:36: I'm gonna go for the Mac OS version.

00:00:38: talking about hardware.

00:00:40: I'm using a Mac mini here from twenty twenty and Apple M one chip only sixteen gigabyte RAM.

00:00:48: So let's see how that goes.

00:00:50: once it is installed on your computer takes like sixty seconds you can Start Olama.

00:00:55: Here we have it.

00:00:57: First thing I want to recommend is that you go into the settings actually.

00:01:02: So you can enable airplane mode.

00:01:04: This then makes a hundred percent sure that no data is sent through the internet, right?

00:01:10: And secondly, you want to take a look at the context length.

00:01:14: The context length basically defines, you know, the amount of memory the LLM has in a given chat.

00:01:20: So how far it can remember back.

00:01:22: So I'd like to go for thirty two K or even more, depending on your hardware.

00:01:27: With that out of the way, we can go back to our main Olama panel and start chatting away, basically.

00:01:35: A quick note on model selection though, so you can select all kinds of open source, open weights models to be precise.

00:01:44: I'm gonna go for Gemma three, hopefully the twelve billion parameter model runs on my hardware here in this demo.

00:01:53: You can select any of those models and when you start a new chat like hello, Olama will automatically start downloading the model if it is not already on your computer.

00:02:05: So now I'm gonna skip ahead until the model is downloaded.

00:02:18: Okay, and now we can see that the model is now downloaded on my computer.

00:02:22: Let's draft a patent, right?

00:02:24: So for this demo, maybe let's say, let's draft the background section of the patent description.

00:02:30: And hey, I've got a full free prompt for this on my website.

00:02:34: So let's head to the browser again.

00:02:38: And let's go to my website, powerclaim.io.

00:02:42: You can go to the free tools.

00:02:46: And here we have the background section writer.

00:02:49: This is a free prompt.

00:02:50: I'm just going to just copy the whole text.

00:02:53: Here we go.

00:02:54: Copy and now back to Olama.

00:02:57: I'm just going to paste the whole prompt into the chat box.

00:03:00: And of course, we also need an invention to work on.

00:03:04: So I'm just going to upload, not upload, but I'm just going to input a sample invention description into my local AI chatbot here and we are submitting the prompt.

00:03:17: Okay, so this took like thirty seconds on my hardware here and now the AI is starting to write out the background section.

00:03:29: As you can see, it's a bit slower than what you might be used to when you're using chat GPT or Gemini.

00:03:37: But of course, if you are following my content, you will not be using a free cloud chatbot for patent drafting, right?

00:03:45: You will never input sensitive information there.

00:03:48: That's why I'm showing you the one hundred percent private option here today.

00:03:54: So it's writing out the background section.

00:03:56: Let's go on a fast forward to the final text.

00:04:04: Okay, and after roughly two minutes, we have the full background section here.

00:04:09: So it starts with a general description of the field.

00:04:13: This invention is about wireless communication networks.

00:04:17: Then it talks about existing technology in that space.

00:04:21: And then it goes through paragraph by paragraph, a number of challenges.

00:04:30: I always write my background sections in the sense that looking at the invention, I have a list of technical effects.

00:04:39: the invention actually achieves right.

00:04:42: and in the background section i like to present exactly the opposite of the technical effect of the invention as a limitation in the known technology.

00:04:52: so here in this background section i've programmed the prompt in such a way that it goes um technical effect by technical effect and writes one paragraph each about a complimentary shortcoming of the prior art.

00:05:09: We have that here.

00:05:10: And then at the end of the background section, we have a closing paragraph that says, you know, developing resource efficient solutions remains a continuing technical problem and so on.

00:05:22: So I think this is a nice springboard then for presenting the invention in the next chapter of the patent application.

00:05:31: marks on model selection.

00:05:33: So in this experiment, I was installing Olama on my very old computer, right with only sixteen gigabytes of RAM, just to show you what is possible.

00:05:44: I was able to run a twelve billion parameter model.

00:05:47: And I think the output was perfectly fine, right?

00:05:54: We have to distinguished between two tasks, though, I would say.

00:05:58: So for the mere text production, drafting a background section, drafting embodiments and things like that, just producing text based on your input.

00:06:08: I like to use models like, for example, the Gemma three on my real computer, which I use productively for my patent attorney work, for my AI drafting in my own practice.

00:06:21: I'm using a Gemma three, the twenty seven billion.

00:06:25: model which is even linguistically even better than in the demo we saw right now.

00:06:31: Or I also like to use GPT OSS, uh, twenty billion parameters.

00:06:37: And then besides tax production, there is a reasoning tasks, right?

00:06:41: So when you use the AI, not only as your, you know, servant to take over the grant work, but also as a tool for thought, as I want to call it.

00:06:50: I have a whole video on this paradigm, which I'm going to link here.

00:06:55: If you use DAI also as a thinking sparing partner, you know, to elevate your own critical thinking rather than replacing it, I like to use reasoning models.

00:07:08: For example, my recommendation would be Quen-III right now.

00:07:13: That's a model I'm using quite frequently in my production machine.

00:07:19: I'm using the thirty billion parameters model here and my production machine is just just a MacBook Pro, so also no magic here.

00:07:28: But of course, when it comes to the linguistic capabilities of those large language models, size does really matter.

00:07:36: So I'm planning to buy a new MacBook, which I think the biggest one has like, one hundred twenty eight gigabytes of RAM, and that should be able to fit even larger local models.

00:07:48: For example, from GPT OSS, I think there's a one hundred twenty billion a version out there and I'm planning to do a test also with those models and I expect even better linguistic quality here.

00:08:02: I'll keep you posted.

00:08:03: Okay, so you've seen a fully private, one hundred percent secure local patent drafting setup in this video.

00:08:11: If you want the exact prompt I was using to create the background section, check the link in the description.

00:08:17: Hope this helps.