LLM

Building a Translation Agent on LlamaEdge

By MileyFu, CNCF Ambassador, DevRel and Founding Member of WasmEdge runtime. Prof. Andrew Ng's agentic translation is a great demonstration on how to cooridnate multiple LLM “agents” to work on a single task. It allows multiple smaller LLMs (like Llama-3 or Gemma-2) to work gether and produce better results than a single large LLM (like ChatGPT). The translation agent is a great fit for LlamaEdge, which provides a lightweight, embeddable, portable, and Docker-native AI runtime for many different types of models and hardware accelerators.…

Getting Started with Gemma-2-9B

Google recently released Gemma 2 models in 9B and 27B Sizes, which are the latest models of its Gemma models family. According to its technical report, there will be an open sourced Gemma-2-2b model in the upcoming days. The technical report also demonstrates that the Gemma-2-9B model outperforms the Mistral-7B, Llama-3-8B, and the Gemma 1.5 models in several benchmarks. In this article, taking Gemma-2-9B as an example, we will cover…

Getting Started with Mistral-7B-Instruct-v0.3

The Mistral-7B-Instruct-v0.3-GGUF model is powered by the innovative GPT architecture, tailored specifically for instructional text understanding, offering unparalleled capabilities in comprehending and generating instructional content. With a vast dataset and rigorous training, Mistral-7B-Instruct-v0.3-GGUF excels in tasks ranging from parsing complex procedural instructions to generating clear and concise instructional texts across various domains. Whether it's guiding users through intricate processes or assisting educators in creating engaging educational materials, this model stands as a pinnacle in the realm of instructional NLP.…

Getting Started with Qwen2-7B-Instruct

Meet Qwen2-7B-Instruct, a powerhouse language model from Alibaba! It's the next generation of Qwen models, boasting serious smarts across various tasks. Compared to previous models, Qwen2-7B-Instruct blows past most open-source options and even competes with secretive proprietary models. This isn't your average language model either. Qwen2-7B-Instruct can handle massive amounts of information, crunching through text up to 131,072 tokens long. That's like tackling a whole book at once! Whether you're working with complex code, trying to solve a mind-bending math problem, or just need some serious language skills, Qwen2-7B-Instruct is ready to impress.…

Getting Started with Codestral-22B-v0.1

Getting Started with Codestral-22B-v0.1 The Codestral-22B-v0.1 is an advanced machine learning model designed to handle a wide array of programming tasks across over 80 programming languages, including popular ones such as Python, Java, C, C++, JavaScript, and Bash. It is specifically tailored for software development, capable of interpreting, documenting, explaining, and refactoring code. The model supports an “instruct” mode which enables it to generate code based on specific instructions, and a “Fill in the Middle” (FIM) mode that predicts missing code tokens between given code snippets.…

Getting Started with Phi-3-mini-128k

The Phi-3-Mini-128K-Instruct is a cutting-edge model with 3.8 billion parameters, designed for lightweight yet powerful natural language processing tasks. Trained on the Phi-3 datasets, which include synthetic and filtered publicly available website data, this model prioritizes high-quality and reasoning-dense properties. It belongs to the Phi-3 family and comes in two variants: 4K and 128K, referring to the context length it can handle in tokens. Following its initial training, the model underwent a rigorous post-training process involving supervised fine-tuning and direct preference optimization.…

Getting Started with Yi-1.5-34B-Chat-16K

On May 20th， Yi released Yi-1.5-9B-Chat-16K and Yi-1.5-34B-Chat-16K, two advanced chat models developed by Yi on Hugging Face. Both models are part of the Yi-1.5 series, which is an improvement over its predecessor, enhancing abilities in areas like coding, math, reasoning, and instruction-following, while maintaining strong language understanding and commonsense reasoning skills. Compared with the Yi-1.5-Chat, the Yi-1.5-9B-Chat-16k has a much longer context window, which means the model can hold longer background information and more complex instructions in the prompt.…

Getting Started with Yi-1.5-9B-Chat

On May 12th, 01.ai released its Yi-1.5 series of models on Hugging Face, which come in 3 sizes: 34/9/6b. Yi-1.5 is a significant upgrade to the previous Yi model. It boasts enhanced capabilities in coding, math, reasoning, and following instructions, while continuing to excel in core language areas like reading comprehension, commonsense reasoning, and understanding language. This advancement is attributed to both a massive dataset of 500 billion tokens for pre-training and fine-tuning on 3 million diverse samples.…

Getting Started with Llama-3-8B

Meta has just released its next generation of open-source LLM, Meta Llama 3. It is the SOTA of LLMs with better performance than the most capable close-source LLMs! Currently, the Llama3 8b and 70b models are available, and a massive 400b model is expected in the next several months. The Llama3 models were trained on a significantly larger dataset compared to its predecessor, Llama 2, resulting in improved capabilities like reasoning and code generation.…

Getting Started with CodeGemma-7b-it

CodeGemma-7b-it is a small yet powerful “coding assistant” model in the Gemma family. It is designed for the following tasks. Code Completion: Imagine you're writing code and get stuck. CodeGemma 7B can analyze the existing code and suggest likely completions, saving you time and effort. Code Generation: Need a whole new block of code for a specific function? CodeGemma 7B can analyze the surrounding code and generate code snippets based on that context.…