Llama-Cpp-Python on akenji's lab

llama-cpp-python - impact of numpy version upgrade

Thu, 04 Jul 2024 00:00:00 +0000

Introduction.

NumPy 2.0.0 was released on June 16. I first noticed it the other day when I tried RAG with using langchain and got an error when building the docker container. Later, I encountered another error in CMake when trying to incorporate llama-cpp-python.

This article summarizes my responses to the two errors I recently experienced.

Background

I recently decided to learn RAG properly, I purchased a japanese book called LLM fine tuning and RAG. The book uses langchain, so I decided to create a docker container for jupyterlab that incorporates the langchain library.

Try the Chatbot UI

Mon, 06 May 2024 00:00:00 +0000

Introduction

In a recent post, I ran the ELYZA 7B model in a local environment using llama-cpp-python. In that post, I mentioned that “about the future” I would like to try to build a system that can chat like ChatGPT.

This time, I built a system that can chat like ChatGPT on a docker container, and I summarize its contents here.

Running Elyza models on GPU using llama-cpp-python

Fri, 03 May 2024 00:00:00 +0000

Motivation

Quantization is essential to run LLM on the local workstation (12-16 GB of GPU memory). In this post, I summarize my attempt to maximize GPU resources using llama-cpp-python.

The content includes some of my mistakes, as I got into some areas due to my lack of understanding.