llama-cpp-agentでローカルLLM+RAGを試す

FROM python:3.10-slim-bullseye

ARG USERNAME=vscode
ARG USER_UID=1000
ARG USER_GID=$USER_UID

ENV LANG ja_JP.UTF-8
ENV LANGUAGE ja_JP:ja
ENV LC_ALL ja_JP.UTF-8
ENV TZ JST-9
ENV TERM xterm

RUN apt-get update \
    && groupadd --gid $USER_GID $USERNAME \
    && useradd -s /bin/bash --uid $USER_UID --gid $USER_GID -m $USERNAME \
    && apt-get install -y sudo \
    && echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME \
    && chmod 0440 /etc/sudoers.d/$USERNAME \
    && apt-get -y install locales \
    && localedef -f UTF-8 -i ja_JP ja_JP.UTF-8

RUN apt install -y build-essential libssl-dev
RUN apt install -y gcc g++

RUN apt -y install cmake
RUN apt-get -y install git

RUN pip install --upgrade pip
RUN pip install --upgrade setuptools

RUN pip install llama-cpp-agent
RUN pip install ragatouille \
    chromadb \
    pip install pysqlite3-binary

__import__('pysqlite3')
import sys
sys.modules['sqlite3'] = sys.modules.pop('pysqlite3')

from ragatouille.utils import get_wikipedia_page
from llama_cpp_agent.messages_formatter import MessagesFormatterType

from typing import List
from pydantic import BaseModel, Field

from llama_cpp_agent.llm_agent import LlamaCppAgent
from llama_cpp_agent.gbnf_grammar_generator.gbnf_grammar_from_pydantic_models import generate_gbnf_grammar_and_documentation
from llama_cpp_agent.rag.rag_colbert_reranker import RAGColbertReranker
from llama_cpp_agent.text_utils import RecursiveCharacterTextSplitter
from llama_cpp_agent.llm_output_settings import LlmStructuredOutputSettings, LlmStructuredOutputType

from llama_cpp import Llama
from llama_cpp_agent.providers import LlamaCppPythonProvider

rag = RAGColbertReranker(persistent=False)
length_function = len
splitter = RecursiveCharacterTextSplitter(
    separators=["\n\n", "\n", " ", ""],
    chunk_size=512,
    chunk_overlap=0,
    length_function=length_function,
    keep_separator=True
)

page = get_wikipedia_page("Synthetic_diamond")
splits = splitter.split_text(page)
for split in splits:
    rag.add_document(split)

llama_model = Llama('/work/models/Phi-3-mini-4k-instruct-q4.gguf', n_ctx=4096)
provider = LlamaCppPythonProvider(llama_model)
settings = provider.get_provider_default_settings()
settings.max_tokens = 4096

class QueryExtension(BaseModel):
    """
    Represents an extension of a query as additional queries.
    """
    queries: List[str] = Field(default_factory=list, description="List of queries.")

grammar, docs = generate_gbnf_grammar_and_documentation([QueryExtension])

query_extension_agent = LlamaCppAgent(
    provider,
    debug_output=True,
    system_prompt="You are a world class query extension algorithm capable of extending queries by writing new queries. Do not answer the queries, simply provide a list of additional queries in JSON format. Structure your output according to the following model:\n\n" + docs.strip(),
    predefined_messages_formatter_type=MessagesFormatterType.PHI_3
)

query = "What is a BARS apparatus?"

output_settings = LlmStructuredOutputSettings.from_pydantic_models([QueryExtension], output_type=LlmStructuredOutputType.object_instance)
output = query_extension_agent.get_chat_response(
    f"Consider the following query: {query}", structured_output_settings=output_settings)

output.queries

['What does BARS stand for in an analytical context?',
 'List applications of BARS in chemistry.',
 'Describe the components and functionality of a typical BARS apparatus.',
 'How is data collected using a BARS apparatus?',
 'Compare BARS with other analytical techniques used for elemental analysis.']

「分析の文脈で BARS とは何を意味しますか?」、「化学における BARS の用途を列挙してください。」、
「一般的な BARS 装置のコンポーネントと機能について説明してください。」、
「BARS 装置を使用してデータはどのように収集されますか?」、
「BARS を元素分析に使用される他の分析手法と比較してください。」

prompt = "Consider the following context:\n==========Context===========\n"
documents = rag.retrieve_documents(query, k=3)
for doc in documents:
    prompt += doc["content"] + "\n\n"

for qu in output.queries:
    documents = rag.retrieve_documents(qu, k=3)
    for doc in documents:
        if doc["content"] not in prompt:
            prompt += doc["content"] + "\n\n"
prompt += "\n======================\nQuestion: " + query

print(prompt)

Consider the following context:
==========Context===========
on the anvils to achieve the same pressure. An alternative is to decrease the surface area to volume ratio of the pressurized volume, by using more anvils to converge upon a higher-order platonic solid, such as a dodecahedron. However, such a press would be complex and difficult to manufacture.

The BARS apparatus is claimed to be the most compact, efficient, and economical of all the diamond-producing presses. In the center of a BARS device, there is a ceramic cylindrical "synthesis capsule" of about 2 

cm3 (0.12 cu in) in size. The cell is placed into a cube of pressure-transmitting material, such as pyrophyllite ceramics, which is pressed by inner anvils made from cemented carbide (e.g., tungsten carbide or VK10 hard alloy). The outer octahedral cavity is pressed by 8 steel outer anvils. After mounting, the whole assembly is locked in a disc-type barrel with a diameter about 1 m (3 ft 3 in). The barrel is filled with oil, which pressurizes upon heating, and the oil pressure is transferred to the central cell. 

These probes consist of a pair of battery-powered thermistors mounted in a fine copper tip. One thermistor functions as a heating device while the other measures the temperature of the copper tip: if the stone being tested is a diamond, it will conduct the tip's thermal energy rapidly enough to produce a measurable temperature drop. This test takes about 2–3 seconds.


== Applications ==


=== Machining and cutting tools ===

Most industrial applications of synthetic diamond have long been associated with 

including lab-grown diamonds within the scope of the definition of "diamond". The revised guide further states that "If a marketer uses 'synthetic' to imply that a competitor's lab-grown diamond is not an actual diamond, ... this would be deceptive." In July 2019, the third party diamond certification lab GIA (Gemological Institute of America) dropped the word 'synthetic' from its certification process and report for lab-grown diamonds, according to the FTC revision.




== Bibliography ==
Barnard, A. S. (2000). The diamond formula: diamond synthesis-a gemological perspective. Butterworth-Heinemann. ISBN 978-0-7506-4244-6.
O'Donoghue, Michael (2006). Gems: their sources, descriptions and identification. Butterworth-Heinemann. ISBN 978-0-7506-5856-0.
Spear, K. E. & Dismukes, J. P. (1994). Synthetic diamond. Wiley-IEEE. ISBN 978-0-471-53589-8.
Lundblad, Erik (1988). Om konsten att göra diamanter. In Daedalus 1988. ISBN 9176160181


== External links ==



In the HPHT method, there are three main press designs used to supply the pressure and temperature necessary to produce synthetic diamond: the belt press, the cubic press and the split-sphere (BARS) press. Diamond seeds are placed at the bottom of the press.


Electronic applications of synthetic diamond are being developed, including high-power switches at power stations, high-frequency field-effect transistors and light-emitting diodes. Synthetic diamond detectors of ultraviolet (UV) light or high-energy particles are used at high-energy research facilities and are available commercially. Due to its unique combination of thermal and chemical stability, low thermal expansion and high optical transparency in a wide spectral range, synthetic diamond is becoming 

For instance, pure diamond is an electrical insulator, but diamond with boron added is an electrical conductor (and, in some cases, a superconductor), allowing it to be used in electronic applications. Nitrogen impurities hinder movement of lattice dislocations (defects within the crystal structure) and put the lattice under compressive stress, thereby increasing hardness and toughness.


=== Thermal conductivity ===


diamonds were chemically identical, their physical properties were not the same. The colorless stones produced strong fluorescence and phosphorescence under short-wavelength ultraviolet light, but were inert under long-wave UV. Among natural diamonds, only the rarer blue gems exhibit these properties. Unlike natural diamonds, all the GE stones showed strong yellow fluorescence under X-rays. The De Beers Diamond Research Laboratory has grown stones of up to 25 carats (5.0 g) for research purposes. Stable 

mining and cutting tools. Polycrystalline diamond is often described by the average size (or grain size) of the crystals that make it up. Grain sizes range from nanometers to hundreds of micrometers, usually referred to as "nanocrystalline" and "microcrystalline" diamond, respectively.


=== Hardness ===
The hardness of diamond is 10 on the Mohs scale of mineral hardness, the hardest known material on this scale. Diamond is also the hardest known natural material for its resistance to indentation. The 


======================
Question: What is a BARS apparatus?

agent_with_rag_information = LlamaCppAgent(
    provider,
    debug_output=True,
    system_prompt="You are an advanced AI assistant, trained by OpenAI. Only answer question based on the context information provided.",
    predefined_messages_formatter_type=MessagesFormatterType.PHI_3
)
agent_output = agent_with_rag_information.get_chat_response(prompt, llm_sampling_settings=settings)
print(agent_output)

prompt = query
agent_with_rag_information = LlamaCppAgent(
    provider,
    debug_output=True,
    system_prompt="You are an advanced AI assistant",
    predefined_messages_formatter_type=MessagesFormatterType.PHI_3
)
agent_output = agent_with_rag_information.get_chat_response(prompt, llm_sampling_settings=settings)
print(agent_output)