Want a powerful large language model right at your fingertips, locally and not in the cloud? Clear about 40GB of storage on your PC and download Nvidia’s AI chatbot for free.
Called Chat with RTX, Nvidia’s AI chatbot is a great way to get accustomed to how large language models work and, in some cases, actually use it for work.
The NVIDIA Chat with RTX tool works like a charm in most cases but does have some drawbacks – mainly that it doesn’t remember. Chat with RTX doesn’t remember your questions, so you won’t be able to do follow-ups and have an actual conversation. However, it’s apparently pretty good at saving time for research.
An editor from The Verge tried it to sift through YouTube videos and found it difficult, as the tool wouldn’t accurately pool information, but said it was very accurate at summarizing.
They also found it great for PDF scanning, something other tools struggle with.
“I’ve also found this useful to scan through PDFs and fact-check data. Microsoft’s own Copilot system doesn’t handle PDFs well within Word, but Nvidia’s Chat with RTX had no problem pulling out all the key information. The responses are near instant as well, with none of the lag you usually see when using cloud-based ChatGPT or Copilot chatbots,” the Verge’s editor wrote.
Nvidia explained in an article how Chat with RTX works:
“Chat with RTX uses retrieval-augmented generation (RAG), NVIDIA TensorRT-LLM software and NVIDIA RTX acceleration to bring generative AI capabilities to local, GeForce-powered Windows PCs. Users can quickly, easily connect local files on a PC as a dataset to an open-source large language model like Mistral or Llama 2, enabling queries for quick, contextually relevant answers.
Rather than searching through notes or saved content, users can simply type queries.
For example, one could ask, “What was the restaurant my partner recommended while in Las Vegas?” and Chat with RTX will scan local files the user points it to and provide the answer with context.”
If you have more than 16GB of RAM and 40GB of storage, plus of course a powerful RTX 30 or 40 series GPU, you can download Chat with RTX and take it for a spin.
Also read: Nvidia Broadcast App Gets Deepfake Update To Make You Look at the Camera
Follow TechTheLead on Google News to get the news first.
