Part 1: Offline GUI-Based Chatbot with Anaconda AI Navigator

1. Title

Accessing a LLM Offline Using Anaconda AI Navigator

2. Objective

The objective of this assignment is to familiarize students with running a large language model (LLM) locally using Anaconda AI Navigator’s built-in graphical user interface (GUI). Students will install Anaconda AI Navigator, test the Phi-3-Mini-4K-Instruct model (or another small model) via the GUI chat interface, configure the built-in API server, and demonstrate its functionality as a basic chatbot application. This lab emphasizes leveraging Anaconda’s out-of-the-box tools for secure, local AI deployment without additional coding.

3. Tools/Resources Needed

• Computer Requirements:
  • OS: Windows 11 or macOS 13+ (Linux not officially supported by Anaconda AI Navigator GUI).
  • RAM: Minimum 8GB, recommended 16GB+ (Phi-3-Mini-4K-Instruct requires ~2-4GB loaded in memory).
  • Disk Space: Minimum 10GB free (Anaconda AI Navigator ~2GB, Phi-3-Mini-4K-Instruct ~4GB with Q4 quantization, plus overhead).
  • CPU/GPU: Multi-core CPU recommended; GPU optional but enhances performance (e.g., NVIDIA with CUDA support).
• Anaconda AI Navigator: Downloadable from the official Anaconda website (requires an Anaconda.com account).
• Phi-3-Mini-4K-Instruct Model: Available via Anaconda AI Navigator’s curated model library (or another small model like TinyLlama if preferred).
• Internet Access: Required for initial download and setup (offline afterward).

4. Description of Assignment

Students will download and install Anaconda AI Navigator, a desktop application providing a GUI for interacting with LLMs locally. Using its built-in features, they will download and test the Phi-3-Mini-4K-Instruct model (a 3.8B parameter model with a 4K token context) or another small model from the library. Students will explore the GUI chat interface by testing the model with at least three distinct prompts directly in the “Chat” tab. Ask the same question a few times, modifying the settings each time and noting the difference.

Next, they will configure the built-in API server (via the “API Server” tab) to expose the model at a local endpoint (e.g., http://localhost:8080), verify its status in Anaconda Navigator, then visit the page, and test out the LLM. This assignment focuses on using Anaconda AI Navigator’s native tools—no external Python coding or libraries beyond what’s provided are required—highlighting a user-friendly approach to local LLM deployment.

5. Deliverables

1. Installation Summary: A brief written report (1 paragraph, ~150 words) documenting the installation process of Anaconda AI Navigator, including any challenges faced (e.g., insufficient disk space) and how you addressed them.
2. Chat Test Output: Screenshots of the GUI chat interface showing responses to at least three different prompts (e.g., “Tell me a story,” “How do you tie a tie?” “Explain local AI”).
3. API Server Configuration: A screenshot of the “API Server” tab showing the running server (e.g., port 8080, model selected) and a screenshot of the “API Explorer” or chat interface confirming a response from the local endpoint.