How to Run and Use AI Locally on Your Low RAM PC

Photo of author

It surely won’t be a bad idea to call this current year an “AI-powered era.” Whether it be our smartphones, PCs, or jobs, we have started seeing the use of artificial intelligence everywhere. However, running the AI-powered feature is still not possible on 100% of the devices as almost all of the AI feature has a minimum hardware requirement.

If you also have an old PC that does not satisfy the minimum system requirement for the latest AI features and tools, don’t worry. There are still a bunch of ways to access them on your low-end devices. In this guide, we’ll whether it’s possible to run LLMs on low-end PCs, if so, then what are these LLM models that are compatible with old PCs?

Can You Run AI Locally on PCs with Low RAM and Old Hardware?

LLM, which stands for Large Language Model, is a tool that is first required to be downloaded locally on the device before running. This is why these tools generally require powerful hardware and high RAM to run smoothly. You can also check the AI capabilities of your PC using the Geekbench AI tool. But anyway, some LLMs do not require any hardware specifications to run.

What are the Specifications Required to Run LLMs?

Well, there is no direct answer to this. The minimum specifications requirement depends on the LLM you are going to use. Some will ask for at least 8GB of RAM while some will ask for 16GB. Most LLMs require your PC to have an eight-core CPU with a powerful in-built or dedicated GPU.

How Can I Run AI on My Old PC?

If your PC does not satisfy the minimum system requirement for most LLMs and you still want to run AI on it, then worry not. We have got you covered. In such cases, you can use Online AI chatbots. These AI tools are as powerful as your locally installed LLMs. You can do almost all the tasks with the online chatbots that you can do with local ones.

But that’s not all we have for you. Local chatbots do have their advantages such as they do not always require an active internet connection or that they can access and know about the details on your device better. The online ones are dependent on the internet though.

Best LLMs to Run on Old Windows PCs

Until now, we have discovered two LLMs that can easily run on low-end devices because they are lightweight. Those are nothing but DistilBERT and ALBERT. However, you cannot use these chatbots for solving complex problems; primarily because they are not meant to be.

DistilBERT is developed was HuggingFace developers. The officials call it a “smaller, faster and cheaper” version of BERT. It is small in size, requires very little memory to run and is probably one of the most efficient small-size LLMs so far.

Another one on our list is ALBERT. It works kind of similarly when compared with the DistilBERT but it is designed very differently. You can also use this LLM if you have a high-end PC but do not want to do any complex tasks.

Further, if your PC has an 8GB of RAM then you can also try the GPT Neo 125M version. It is an open-source version and can be adjusted and customized according to the user’s want. It makes a good balance of performance and system requirements. It performs as good as GPT-2, if not better.

How to Install LLM Chatbots

Before anything, all the LLMs have different installation methods so we cannot exactly point out a work-for-all method. But before moving on to the download and installation phase, make sure to check the hardware of your PC and look for a compatible LLM. You can then download a tool called Docker. This tool will allow you to run the application in special boxes.

Now look for LLM software on the source website, be it HuggingFace or GitHub. Read the installation instructions and follow them step by step. You may be required to download a software update later as well so keep an eye on that. You can also check the respective LLMs community on Reddit or any other platform for more guidance and help if needed.

Related Articles:

Explore Stories

AUTHOR.

Photo of author

Chandan

Iโ€™m a consumer tech writer passionate about breaking down the latest gadgets and smartphones into easy-to-understand guides and news. I love exploring new tech firsthand and sharing practical, relatable insights to help readers stay ahead in the ever-evolving world of technology.

Leave a Comment

Share to...