GPU

Introduction.

NumPy 2.0.0 was released on June 16. I first noticed it the other day when I tried RAG with using langchain and got an error when building the docker container. Later, I encountered another error in CMake when trying to incorporate llama-cpp-python.

This article summarizes my responses to the two errors I recently experienced.

Dealing with errors related to NumPy 2.0.0

Background

I recently decided to learn RAG properly, I purchased a japanese book called LLM fine tuning and RAG. The book uses langchain, so I decided to create a docker container for jupyterlab that incorporates the langchain library.

Introduction

In a recent post, I ran the ELYZA 7B model in a local environment using llama-cpp-python. In that post, I mentioned that “about the future” I would like to try to build a system that can chat like ChatGPT.

This time, I built a system that can chat like ChatGPT on a docker container, and I summarize its contents here.

Motivation

Quantization is essential to run LLM on the local workstation (12-16 GB of GPU memory). In this post, I summarize my attempt to maximize GPU resources using llama-cpp-python.

The content includes some of my mistakes, as I got into some areas due to my lack of understanding.

Introduction

About a month ago, I wrote an article about building a GPU cluster using kubernetes in this article. At that time, the GPU pod was in Pending state and did not work. After that, I managed to get it to work thanks to the advice of a certain person, so I’ll summarize it here.

In my environment, there is a problem that the GPU pod does not start up until a certain node is started, and furthermore, I have not been able to specify GPUs in a node, specify nodes, and so on, which is what I assumed, so I decided to call it “the first step”.

Motivation

I took on the challenge of creating a GPU cluster using multiple workstations I have set up at home to study Kubernetes.

It was a hurdle for me, as I was new to Kubernetes. This is because I had to learn from the documentation whether the operations were necessary only during installation or when building the cluster (during operation).

As it stands, the GPU cluster is not operational!　This document is incomplete.