Run ChatGPT-Like LLM Models offline on your PC

Ever since ChatGPT hyped the use of AI, everyone wanted to try similar AI models on their devices for daily queries. These are powerful AI models capable of executing high-level tasks. Now with the help of customized tools, users can install various Large Language Models (LLMs) on their computers for offline use.

If you want to make your computer an AI chat tool that you can access anytime you want, then the answer is the tool that allows users to install various AI models such as Llama 3, Phi 3, Falcon, Gemma, Replit, and more. Let’s now see the steps to get these tools and setup AI Models.

Minimum Requirements:

  • Apple Silicon Mac (M1/M2/M3) with macOS 13.6 or newer
  • Windows / Linux PC with a processor that supports AVX2 (typically newer PCs)
  • 16GB+ of RAM is recommended. For PCs, 6GB+ of VRAM is recommended
  • NVIDIA/AMD GPUs supported

Note: You can follow this guide for Windows, Mac, and Linux platforms. AI Models use a high amount of RAM, CPU, and GPU. If your device has fewer resources, it will take a long time to respond to your queries.

How to Install LLM Locally on your PC

Step 1: First download and install the LM Studio software on your respective platform. The tool is around 400MB in size before installation.

Step 2: After installing the tool, launch it. It will open the home screen of the tool with the Search bar.

Install LLMs on PC

Step 3: In the Search bar you can search for various LLMs, that you want to install on your PC. Let’s say you want to install Llama3 from Meta AI, type Llama and press enter.

Step 4: It will show a list of available repositories related to your query. Click on the Llama2 or Llama3. And it will expand the models from that repository on the right-hand side.

Install LLMs on PC

Step 5: The result will show all the details about the model, like Size, and GPU offload. Click on the Download button next to the model you want to install on your PC.

Step 6: To see the ongoing download, click on the Arrow facing upward at the bottom right corner.

Step 7: After download, it will automatically add to your collection. Once it’s done, you will see the Complete status.

Install LLMs on PC

Step 8: Now tap the AI Chat icon (bubble icon) from the side menu. Here you need to select the Model that you downloaded. To do that tap the top bar and select the model.

Install LLMs on PC

Step 9: It will load the selected model which can take a few seconds. Once it is loaded, you will be able to chat with AI just like Chat GPT.

You can ask any query. The response time will depend on your PC hardware and software capability. If you have high end PCs, you can expect better performance even in the case of bigger AI Models. Since all the operations take place locally on your PC, everything remains private. In case you have any queries, you can ask in the comment section below.

Also Check:

Explore Stories

AUTHOR.

Photo of author

Ashutosh singh

Computer Science graduate and tech enthusiast. Love testing new tech gadgets ever since my first windows phone from Samsung. Working as a technical writer for more than five years, and I believe I have also developed good SEO skills. In my spare time, I like watching movies, and playing games.

Leave a Comment