Run ChatGPT-Like LLM Models offline on your PC

By Ashwin Ananthapadmanabhan

       Ever since ChatGPT hyped the use of AI, everyone wanted to try similar AI models on their devices for daily queries. These are powerful AI models capable of executing high-level tasks. Now with the help of customized tools, users can install various Large Language Models (LLMs) on their computers for offline use.

If you want to make your computer an AI chat tool that you can access anytime you want, then the answer is the tool that allows users to install various AI models such as Llama 3, Phi 3, Falcon, Gemma, Replit, and more. Let’s now see the steps to get these tools and setup AI Models.

Minimum Requirements:

  1. Apple Silicon Mac (M1/M2/M3) with macOS 13.6 or newer
  2. Windows / Linux PC with a processor that supports AVX2 (typically newer PCs)
  3. 16GB+ of RAM is recommended. For PCs, 6GB+ of VRAM is recommended
  4. NVIDIA/AMD GPUs supported

Note: You can follow this guide for Windows, Mac, and Linux platforms. AI Models use a high amount of RAM, CPU, and GPU. If your device has fewer resources, it will take a long time to respond to your queries.

How to Install LLM Locally on your PC:

  1. First download and install the LM Studio software on your respective platform. The tool is around 400MB in size before installation.
  2. After installing the tool, launch it. It will open the home screen of the tool with the Search bar.
  3. In the Search bar you can search for various LLMs, that you want to install on your PC. Let’s say you want to install Llama3 from Meta AI, type Llama and press enter.
  4. It will show a list of available repositories related to your query. Click on the Llama2 or Llama3. And it will expand the models from that repository on the right-hand side.
  5. The result will show all the details about the model, like Size, and GPU offload. Click on the Download button next to the model you want to install on your PC.
  6. To see the ongoing download, click on the Arrow facing upward at the bottom right corner.
  7. After download, it will automatically add to your collection. Once it’s done, you will see the Complete status.
  8. Now tap the AI Chat icon (bubble icon) from the side menu. Here you need to select the Model that you downloaded. To do that tap the top bar and select the model.
  9. It will load the selected model which can take a few seconds. Once it is loaded, you will be able to chat with AI just like Chat GPT.
You can ask any query. The response time will depend on your PC hardware and software capability. If you have high end PCs, you can expect better performance even in the case of bigger AI Models. Since all the operations take place locally on your PC, everything remains private. 


For more please refer the official help doc & blog !

Comments

Post a Comment

Popular posts from this blog

LOADING TIME OF YOUR WEBSITE

By Ashwin Ananthapadmanabhan
Image
CHECK LOADING TIME OF YOUR WEBSITE Now a day’s everyone is using Google to search their information. There are millions of websites are available in Google, each websites is with its own content.  When you search any information in Google, it will list the links, which is related to your topic. Each link is directed to some websites. While searching, the  user always  wants to load the websites very fast. Because now a days, in this  busy world no one has the patience. If the sites are loading slowly then the user may get tensed. So it may lead the people to get bore while using your website. If you are the blogger you should focus on your web page loading time. Loading time is very important goal to run a successful website.  So if you want to attract the visitors to your site you have to reduce the loading time of your website. Here are some websites to check your site loading speed freely. So you can easily analyze
Read more

How To Watch Time-lapse In Google Earth: Largest Video On The Planet!

By Ashwin Ananthapadmanabhan
Image
It’s a good way to see the world. Representative image. Image by Greg Rakozy on Unsplashed      In the ‘biggest update to Google Earth since 2017,’ Google has added the new Time-lapse feature. Time-lapse in Google Earth combines 24 million satellite photos from the past 37 years to make a 4D interactive model. As the name suggests, the Google Earth Time-lapse feature lets you go through a time-lapse of how things have changed over the years.      Time-lapse in Google Earth consists of photos taken from 1984 to 2020, which equals 20 petabytes (1 Petabyte is one million gigabytes) of satellite images. Google took these pictures and stitched them into a 4.4 terapixel video. The company says that this entire video is equivalent to over 5 million 4K resolution videos. That said, Google is also calling Time-lapse on Google Earth the largest video on the planet.  In its blog  post , the company has also promised annual updates to Google Earth for the next decade. You can use Google Time-lapse
Read more