import sys

print(sys.executable)

/Users/mohitsalur/Documents/AtoZ/Causify_AI/gpsaggese.github.io/tutorials/tutorial_llamaindex/venv/bin/python

LlamaIndex API Overview¶

This notebook demonstrates the core API components of LlamaIndex and how they are composed to build a retrieval-augmented generation (RAG) pipeline.

We focus on:

Model configuration
Document loading
Node parsing (chunking)
Index construction
Query engine configuration
Prompt customization

This notebook uses open-source models and does not require API keys.

from llama_index.core import (
    VectorStoreIndex,
    SimpleDirectoryReader,
    Settings,
)
from llama_index.core.node_parser import SentenceSplitter
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.huggingface import HuggingFaceLLM
from llama_index.core.prompts import PromptTemplate

import os

/Users/mohitsalur/Documents/AtoZ/Causify_AI/gpsaggese.github.io/tutorials/tutorial_llamaindex/venv/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Model Configuration¶

LlamaIndex separates responsibilities between:

Embedding model: Converts text into vector representations for similarity search.
LLM (Language Model): Generates responses based on retrieved context.
Settings: Global configuration registry that wires components together.

We configure both the embedding model and the LLM below.

embed_model = HuggingFaceEmbedding(
    model_name="sentence-transformers/all-MiniLM-L6-v2"
)

Settings.embed_model = embed_model

print("Embedding model loaded successfully.")

Embedding model loaded successfully.

Language Model Configuration¶

We use an open-source, instruction-tuned causal language model:

TinyLlama-1.1B-Chat

This model:

Is small enough to run locally
Supports instruction-style prompts
Works with AutoModelForCausalLM
Does not require API keys

We register it globally using the Settings object.

llm = HuggingFaceLLM(
    model_name="TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    tokenizer_name="TinyLlama/TinyLlama-1.1B-Chat-v1.0",
    context_window=2048,
    max_new_tokens=128,
    generate_kwargs={"temperature": 0.0},
    device_map="auto",
)

Settings.llm = llm

print("LLM loaded successfully.")

LLM loaded successfully.

Document Loading¶

Documents are the primary input abstraction in LlamaIndex.

SimpleDirectoryReader:

Reads files from a directory
Wraps them into Document objects
Prepares them for parsing and indexing

For demonstration purposes, we create a small toy dataset.

# Create demo directory
os.makedirs("data/api_demo", exist_ok=True)

# Write a small sample file
with open("data/api_demo/sample.txt", "w") as f:
    f.write(
        "LlamaIndex is a framework for building LLM-powered applications. "
        "It supports document ingestion, indexing, and retrieval. "
        "Retrieval-augmented generation improves factual grounding."
    )

# Load documents
documents = SimpleDirectoryReader("data/api_demo").load_data()

print(f"Number of documents loaded: {len(documents)}")
documents

Number of documents loaded: 1

[Document(id_='2ec74507-9798-48ac-b1b0-e6a9845c23b0', embedding=None, metadata={'file_path': '/Users/mohitsalur/Documents/AtoZ/Causify_AI/gpsaggese.github.io/tutorials/tutorial_llamaindex/data/api_demo/sample.txt', 'file_name': 'sample.txt', 'file_type': 'text/plain', 'file_size': 180, 'creation_date': '2026-03-02', 'last_modified_date': '2026-03-02'}, excluded_embed_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], excluded_llm_metadata_keys=['file_name', 'file_type', 'file_size', 'creation_date', 'last_modified_date', 'last_accessed_date'], relationships={}, metadata_template='{key}: {value}', metadata_separator='\n', text_resource=MediaResource(embeddings=None, data=None, text='LlamaIndex is a framework for building LLM-powered applications. It supports document ingestion, indexing, and retrieval. Retrieval-augmented generation improves factual grounding.', path=None, url=None, mimetype=None), image_resource=None, audio_resource=None, video_resource=None, text_template='{metadata_str}\n\n{content}')]

Node Parsing (Chunking)¶

Before indexing, documents are split into smaller units called nodes.

Chunking is important because:

Embeddings operate on smaller text segments
Retrieval happens at the node level
Chunk size affects retrieval quality and context precision

We demonstrate custom chunk configuration below.

# Configure chunking behavior
parser = SentenceSplitter(
    chunk_size=100,
    chunk_overlap=20,
)

# Parse documents into nodes
nodes = parser.get_nodes_from_documents(documents)

print(f"Number of nodes created: {len(nodes)}")
print("\nFirst node text:\n")
print(nodes[0].text)

Number of nodes created: 1

First node text:

LlamaIndex is a framework for building LLM-powered applications. It supports document ingestion, indexing, and retrieval. Retrieval-augmented generation improves factual grounding.

Index Construction¶

VectorStoreIndex:

Computes embeddings for each node
Stores them in a vector index
Enables similarity-based retrieval

The index operates at the node level.

index = VectorStoreIndex(nodes)

print("Index constructed successfully.")

Index constructed successfully.

Query Engine¶

The Query Engine coordinates:

Query embedding
Similarity search over indexed nodes
Context injection into a prompt
Response generation via the LLM

We configure retrieval parameters such as similarity_top_k.

query_engine = index.as_query_engine(similarity_top_k=1)

response = query_engine.query("What does LlamaIndex support?")

print(response)

The following generation flags are not valid and may be ignored: ['temperature']. Set `TRANSFORMERS_VERBOSITY=info` for more details.

LlamaIndex supports document ingestion, indexing, and retrieval.

Inspect Retrieved Source Nodes¶

For transparency, LlamaIndex allows inspection of retrieved source nodes.

This is useful for:

Debugging retrieval quality
Understanding context injection
Evaluating RAG behavior

for i, source_node in enumerate(response.source_nodes):
    print(f"--- Retrieved Node {i + 1} ---\n")
    print(source_node.node.text)
    print("\nScore:", source_node.score)
    print("\n")

--- Retrieved Node 1 ---

LlamaIndex is a framework for building LLM-powered applications. It supports document ingestion, indexing, and retrieval. Retrieval-augmented generation improves factual grounding.

Score: 0.620426523732702

Custom Prompt Template¶

LlamaIndex allows customization of how retrieved context and queries are formatted before being sent to the LLM.

This is useful for:

Adjusting instruction style
Improving response formatting
Controlling output structure

custom_prompt = PromptTemplate(
    "You are a technical assistant.\n\n"
    "Context:\n{context_str}\n\n"
    "Question: {query_str}\n"
    "Answer concisely:"
)

query_engine_custom = index.as_query_engine(
    similarity_top_k=1,
    text_qa_template=custom_prompt,
)

response_custom = query_engine_custom.query(
    "Explain retrieval-augmented generation."
)

print(response_custom)

Retrieval-augmented generation is a technique that enhances the quality of generated text by incorporating information from the retrieved documents. It is a form of natural language generation that uses the information from the retrieved documents to improve the quality of the generated text.

Summary¶

In this notebook, we demonstrated the core API components of LlamaIndex:

Model configuration via Settings
Document ingestion with SimpleDirectoryReader
Node parsing (chunking)
Vector index construction
Query engine orchestration
Retrieval transparency
Prompt customization

This modular design allows flexible construction of retrieval-augmented generation systems.

The next notebook demonstrates a complete end-to-end example using real-world data.