diff --git a/deployment/on-device/atomic-chat.mdx b/deployment/on-device/atomic-chat.mdx new file mode 100644 index 0000000..f730e53 --- /dev/null +++ b/deployment/on-device/atomic-chat.mdx @@ -0,0 +1,169 @@ +--- +title: "Atomic Chat" +description: "Atomic Chat is a desktop and mobile app for running LLMs locally with a graphical user interface." +--- + + + Use Atomic Chat for local inference with a graphical interface on desktop and mobile, one-click model downloads from Hugging Face, and no command-line setup. + + +Atomic Chat uses GGUF models on all platforms and MLX models on Apple Silicon. + +## Installation + +Download and install Atomic Chat from [atomic.chat](https://atomic.chat): + +* **macOS** (Apple Silicon): DMG installer +* **Windows** (x64): EXE installer +* **Linux** (x86_64): AppImage +* **iPhone and iPad**: [App Store](https://apps.apple.com/us/app/atomic-chat-private-local-ai/id6761720226) +* **Android**: [Google Play](https://play.google.com/store/apps/details?id=chat.atomic.app) + +## Downloading Models + +1. Open Atomic Chat and open the model library via the **Models** tab +2. Search for "LiquidAI" +3. Select a model and quantization level (`Q4_K_M` recommended) +4. Click **Download** + +Alternatively, enable [Hugging Face Local Apps](https://huggingface.co/docs/hub/local-apps), then choose **Use this model** > **Atomic Chat** from a compatible model page. + +See the [Models page](/lfm/models/complete-library) for all available GGUF models. + +## Using the Chat Interface + +1. Go to the **New Chat** tab +2. Select your model from the dropdown +3. Adjust parameters (`temperature`, `top_k`, `repeat_penalty`) in the model settings +4. Start chatting + +## Generation Parameters + +Control text generation behavior using the GUI sidebar or API parameters. Key parameters: + +* **`temperature`** (`float`, default 1.0): Controls randomness (0.0 = deterministic, higher = more random). Typical range: 0.1-2.0 +* **`top_p`** (`float`, default 1.0): Nucleus sampling - limits to tokens with cumulative probability ≤ top\_p. Typical range: 0.1-1.0 +* **`top_k`** (`int`, default 40): Limits to top-k most probable tokens. Typical range: 1-100 +* **`repeat_penalty`** (`float`, default 1.0): Penalty for repeating tokens (>1.0 = discourage repetition). Typical range: 1.0-1.5 + +Via the OpenAI-compatible API: + +```python +response = client.chat.completions.create( + model="", + messages=[{"role": "user", "content": "What is machine learning?"}], + temperature=0.1, + max_tokens=512, + extra_body={"top_k": 50, "repeat_penalty": 1.05}, +) +``` + +## Running the Server + +On desktop, Atomic Chat can serve the currently loaded model through a local OpenAI-compatible server for programmatic access: + +1. Load the model in a chat +2. Open the **Integrations** tab +3. Click **Start Server**. The server defaults to `http://localhost:1337/`; use the port shown in the app if you changed it or if that port was unavailable. + +Get the loaded model's ID before sending requests: + +```bash +curl http://localhost:1337/v1/models +``` + +Use the OpenAI Python client: + +```python +from openai import OpenAI + +client = OpenAI( + base_url="http://localhost:1337/v1", + api_key="not-needed" +) + +response = client.chat.completions.create( + model="", + messages=[ + {"role": "user", "content": "What is machine learning?"} + ], + temperature=0.1, + max_tokens=512, + extra_body={"top_k": 50, "repeat_penalty": 1.05}, +) +print(response.choices[0].message.content) +``` + +### Streaming Responses + +```python +stream = client.chat.completions.create( + model="", + messages=[ + {"role": "user", "content": "Tell me a story."} + ], + stream=True +) + +for chunk in stream: + if chunk.choices[0].delta.content is not None: + print(chunk.choices[0].delta.content, end="") +``` + +You can also use curl to interact with the server: + + + ```bash + curl http://localhost:1337/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "", + "messages": [{"role": "user", "content": "Hello!"}], + "temperature": 0.1, + "top_k": 50, + "repeat_penalty": 1.05 + }' + ``` + + +## Vision Models + +Atomic Chat supports LFM2-VL and LFM2.5-VL GGUF models on desktop. Its mobile catalog is curated by platform; LFM2.5-VL-1.6B is available for mobile vision inference. + +Download a vision model from the model library, then attach images to your messages to ask questions about them. + + + ```python + from openai import OpenAI + import base64 + + client = OpenAI( + base_url="http://localhost:1337/v1", + api_key="not-needed" + ) + + # Encode image to base64 + with open("image.jpg", "rb") as image_file: + image_data = base64.b64encode(image_file.read()).decode("utf-8") + + response = client.chat.completions.create( + model="", + messages=[ + { + "role": "user", + "content": [ + {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_data}"}}, + {"type": "text", "text": "What's in this image?"} + ] + } + ] + ) + print(response.choices[0].message.content) + ``` + + +## Tips + +* **Quantization**: `Q4_K_M` offers the best balance of size and quality; step up to `Q6_K` or `Q8_0` if you have memory to spare +* **Apple Silicon**: GGUF models run with Metal acceleration, and MLX builds of LFM models are supported natively +* **Long conversations (desktop)**: TurboQuant can compress the KV cache to 3 or 4 bits, so long contexts fit in significantly less memory diff --git a/docs.json b/docs.json index 5533453..76c73fb 100644 --- a/docs.json +++ b/docs.json @@ -97,7 +97,8 @@ "deployment/on-device/lm-studio", "deployment/on-device/mlx", "deployment/on-device/onnx", - "deployment/on-device/ollama" + "deployment/on-device/ollama", + "deployment/on-device/atomic-chat" ] }, { diff --git a/examples/agent-harnesses.mdx b/examples/agent-harnesses.mdx index eb93ea4..7a42141 100644 --- a/examples/agent-harnesses.mdx +++ b/examples/agent-harnesses.mdx @@ -16,7 +16,7 @@ local URL yours prints. You point your harness at that URL. Each backend uses its own default port, so your endpoint depends on which one you run. - llama.cpp and MLX use `8080`, vLLM uses `8000`, SGLang uses `30000`, and LM Studio uses `1234`. + llama.cpp and MLX use `8080`, vLLM uses `8000`, SGLang uses `30000`, LM Studio uses `1234`, and Atomic Chat uses `1337`. The examples in this guide use `http://localhost:8080/v1`. When you configure a harness, replace the port with your server's. @@ -101,6 +101,21 @@ Install one backend and start it with tool calling enabled. 3. Set the context length in the model settings. 4. Click **Start Server**. It serves at `http://localhost:1234`. + + **Install:** + + Download and install [Atomic Chat](https://atomic.chat), then search for **LFM2.5-2.6B** in + the model library and download the `Q4_K_M` GGUF. See the + [Atomic Chat guide](/deployment/on-device/atomic-chat). + + **Run:** + + 1. Load **LFM2.5-2.6B** in a chat. + 2. Open the **Integrations** tab. + 3. Click **Start Server**. It serves the loaded model at `http://localhost:1337` by default; use the port shown in the app. + + The same Integrations screen also includes one-click agent launchers, so you can run a harness right next to the server. + **Install** (Apple Silicon only): diff --git a/guides/migration-guide.mdx b/guides/migration-guide.mdx index 709d9be..579faca 100644 --- a/guides/migration-guide.mdx +++ b/guides/migration-guide.mdx @@ -36,7 +36,7 @@ Across runtimes, avoid carrying over hand-written Qwen, Llama, or Gemma prompt t For GPU inference, see [Transformers](/deployment/gpu-inference/transformers), [vLLM](/deployment/gpu-inference/vllm), or [SGLang](/deployment/gpu-inference/sglang). -For edge and on-device inference, see [llama.cpp](/deployment/on-device/llama-cpp), [Ollama](/deployment/on-device/ollama), [LM Studio](/deployment/on-device/lm-studio), [MLX](/deployment/on-device/mlx), [ONNX](/deployment/on-device/onnx), or the [LEAP SDK](/deployment/on-device/sdk/quick-start). +For edge and on-device inference, see [llama.cpp](/deployment/on-device/llama-cpp), [Ollama](/deployment/on-device/ollama), [Atomic Chat](/deployment/on-device/atomic-chat), [LM Studio](/deployment/on-device/lm-studio), [MLX](/deployment/on-device/mlx), [ONNX](/deployment/on-device/onnx), or the [LEAP SDK](/deployment/on-device/sdk/quick-start). ## Chat Template diff --git a/guides/use-case-evaluation.mdx b/guides/use-case-evaluation.mdx index ec73c4b..65b252d 100644 --- a/guides/use-case-evaluation.mdx +++ b/guides/use-case-evaluation.mdx @@ -15,7 +15,7 @@ Use this guide when you want to validate LFMs on an actual product task. The goa ## Integration -LFMs serve through OpenAI-compatible endpoints on [vLLM](/deployment/gpu-inference/vllm) and [SGLang](/deployment/gpu-inference/sglang), load in [Transformers](/deployment/gpu-inference/transformers), and run locally through [llama.cpp](/deployment/on-device/llama-cpp), [Ollama](/deployment/on-device/ollama), [MLX](/deployment/on-device/mlx), and [LM Studio](/deployment/on-device/lm-studio). +LFMs serve through OpenAI-compatible endpoints on [vLLM](/deployment/gpu-inference/vllm) and [SGLang](/deployment/gpu-inference/sglang), load in [Transformers](/deployment/gpu-inference/transformers), and run locally through [llama.cpp](/deployment/on-device/llama-cpp), [Ollama](/deployment/on-device/ollama), [Atomic Chat](/deployment/on-device/atomic-chat), [MLX](/deployment/on-device/mlx), and [LM Studio](/deployment/on-device/lm-studio). Read these three pages before judging quality: diff --git a/lfm/help/faqs.mdx b/lfm/help/faqs.mdx index 880d8c4..31c635f 100644 --- a/lfm/help/faqs.mdx +++ b/lfm/help/faqs.mdx @@ -53,7 +53,7 @@ Yes! Use the [LEAP SDK](/deployment/on-device/sdk/quick-start) to deploy models -- **GGUF**: For llama.cpp, LM Studio, Ollama (Q4_0, Q4_K_M, Q5_K_M, Q6_K, Q8_0, F16) +- **GGUF**: For llama.cpp, LM Studio, Ollama, Atomic Chat (Q4_0, Q4_K_M, Q5_K_M, Q6_K, Q8_0, F16) - **MLX**: For Apple Silicon (4-bit, 5-bit, 6-bit, 8-bit, bf16) - **ONNX**: For cross-platform deployment with ONNX Runtime diff --git a/lfm/models/complete-library.mdx b/lfm/models/complete-library.mdx index de27764..19dbc00 100644 --- a/lfm/models/complete-library.mdx +++ b/lfm/models/complete-library.mdx @@ -8,7 +8,7 @@ description: "Liquid Foundation Models (LFMs) are a new class of multimodal arch All of our models share the following capabilities: - 32K token context length for extended conversations and document processing (128K for LFM2.5-8B-A1B) -- Designed for fast inference with [Transformers](/deployment/gpu-inference/transformers), [llama.cpp](/deployment/on-device/llama-cpp), [vLLM](/deployment/gpu-inference/vllm), [SGLang](/deployment/gpu-inference/sglang), [MLX](/deployment/on-device/mlx), [Ollama](/deployment/on-device/ollama), and [LEAP](/deployment/on-device/sdk/quick-start) +- Designed for fast inference with [Transformers](/deployment/gpu-inference/transformers), [llama.cpp](/deployment/on-device/llama-cpp), [vLLM](/deployment/gpu-inference/vllm), [SGLang](/deployment/gpu-inference/sglang), [MLX](/deployment/on-device/mlx), [Ollama](/deployment/on-device/ollama), [Atomic Chat](/deployment/on-device/atomic-chat), and [LEAP](/deployment/on-device/sdk/quick-start) - Trainable via SFT, DPO, VLM, and GRPO workflows with [LEAP Finetune](/lfm/fine-tuning/leap-finetune), [TRL](/lfm/fine-tuning/trl), and [Unsloth](/lfm/fine-tuning/unsloth) @@ -53,7 +53,7 @@ Start with the model family that matches your input and output shape, then choos - Use [llama.cpp](/deployment/on-device/llama-cpp), [Ollama](/deployment/on-device/ollama), [MLX](/deployment/on-device/mlx), or the [LEAP SDK](/deployment/on-device/sdk/quick-start) depending on platform and packaging needs. + Use [llama.cpp](/deployment/on-device/llama-cpp), [Ollama](/deployment/on-device/ollama), [Atomic Chat](/deployment/on-device/atomic-chat), [MLX](/deployment/on-device/mlx), or the [LEAP SDK](/deployment/on-device/sdk/quick-start) depending on platform and packaging needs. @@ -70,8 +70,8 @@ Start with the model family that matches your input and output shape, then choos Use the format that matches your runtime and deployment target: -- **GGUF** — Best for local CPU/GPU inference on any platform. Use with [llama.cpp](/deployment/on-device/llama-cpp), [LM Studio](/deployment/on-device/lm-studio), or [Ollama](/deployment/on-device/ollama). Append `-GGUF` to any model name. -- **MLX** — Best for Mac users with Apple Silicon. Leverages unified memory for fast inference via [MLX](/deployment/on-device/mlx). Browse at [mlx-community](https://huggingface.co/mlx-community/collections?search=LFM). +- **GGUF** — Best for local CPU/GPU inference on any platform. Use with [llama.cpp](/deployment/on-device/llama-cpp), [LM Studio](/deployment/on-device/lm-studio), [Ollama](/deployment/on-device/ollama), or [Atomic Chat](/deployment/on-device/atomic-chat). Append `-GGUF` to any model name. +- **MLX** — Best for Mac users with Apple Silicon. Leverages unified memory for fast inference via [MLX](/deployment/on-device/mlx) or [Atomic Chat](/deployment/on-device/atomic-chat). Browse at [mlx-community](https://huggingface.co/mlx-community/collections?search=LFM). - **ONNX** — Best for production deployments and edge devices. Cross-platform with ONNX Runtime across CPUs, GPUs, and accelerators. Append `-ONNX` to any model name. ### Quantization diff --git a/link-snapshot.yaml b/link-snapshot.yaml index f68f271..262f275 100644 --- a/link-snapshot.yaml +++ b/link-snapshot.yaml @@ -30,6 +30,7 @@ active: - /deployment/on-device/android/openai-client - /deployment/on-device/android/utilities - /deployment/on-device/android/voice-assistant + - /deployment/on-device/atomic-chat - /deployment/on-device/ios/advanced-features - /deployment/on-device/ios/ai-agent-usage-guide - /deployment/on-device/ios/cloud-ai-comparison