0. Setup and Initialization¶

Before diving into use cases, we should initialize the notebook with the required setup:

# Import the helper script.
import helpers.hllm as hllm
import pandas as pd

# Set up logging for debugging.
import logging

logging.basicConfig(level=logging.INFO)

# Set OpenAI API key.
import os
from typing import List, Tuple

os.environ["OPENAI_API_KEY"] = "<your_api_key_here>"

1. Travel Agent chat assistant:¶

Goal: Cretae a chat agent that will help the user to create an itinary to visit New York Trip considering all the constraints.¶

# Define the prompt for the travel assistant
user_prompt = """
I am visiting New York City for 3 days. Please create a detailed itinerary,
including popular attractions, food recommendations, and some evening activities.
I already booked flight tickets and hotel near Newark penn station.
Constraints:
1) Dates: from 24th to 27th Dec.
1) My budget for travel is around $400 excluding hotel and flight.
2) I am planning to travel through subway and for rest of the trip I am planning to walk.
3) Also, take into account traffic and tourist rush at popular places.
"""

# Define the system instructions for the assistant
system_instructions = """
You are a travel assistant specializing in creating personalized travel itineraries.
Your recommendations should balance sightseeing, food, and leisure activities considering provided constraints.
Provide details like the time required for activities and approximate costs where possible.
"""

# Use the get_completion method to generate the trip plan
trip_plan = hllm.get_completion(
    user_prompt=user_prompt,
    system_prompt=system_instructions,
    model="gpt-4o-mini",
    temperature=0.7,  # Slightly increase temperature for creative outputs
)

# Print the generated trip itinerary
print("3-Day New York City Trip Itinerary:")
print(trip_plan)

LLM API call ...

INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"

LLM API call done (22.638 s)
3-Day New York City Trip Itinerary:
Here's a detailed itinerary for your 3-day trip to New York City from December 24th to December 27th, considering your budget, transportation preferences, and the holiday season's traffic and tourist rush. 

---

### Day 1: December 24th (Christmas Eve)

**Morning:**

- **Breakfast at a local diner:** 
  - *Recommendation:* Tick Tock Diner (near Times Square)
  - *Cost:* ~$15
  - *Time:* 1 hour
  
- **Visit Central Park:**
  - *Activity:* Enjoy a stroll through Central Park. Make sure to see iconic spots like Bethesda Terrace and the Bow Bridge.
  - *Cost:* Free
  - *Time:* 2 hours

**Afternoon:**

- **Lunch in the Upper West Side:**
  - *Recommendation:* The Meatball Shop (casual dining)
  - *Cost:* ~$15
  - *Time:* 1 hour
  
- **The Metropolitan Museum of Art:**
  - *Activity:* Explore one of the world's largest and finest art museums.
  - *Cost:* Suggested donation is $25, but you can pay what you wish.
  - *Time:* 2-3 hours

**Evening:**

- **Dinner in the East Village:**
  - *Recommendation:* Momofuku Noodle Bar (famous ramen)
  - *Cost:* ~$20
  - *Time:* 1 hour
  
- **Visit Rockefeller Center:**
  - *Activity:* See the famous Christmas tree and ice skating rink.
  - *Cost:* Free (ice skating costs ~$20)
  - *Time:* 1-2 hours
  
- **Catch a Broadway Show:**
  - *Recommendation:* Check for last-minute tickets at TKTS for a matinee or evening performance.
  - *Cost:* ~$50-$100 depending on the show
  - *Time:* 2-3 hours

---

### Day 2: December 25th (Christmas Day)

**Morning:**

- **Christmas Breakfast:**
  - *Recommendation:* Ess-a-Bagel (try their famous bagels)
  - *Cost:* ~$10
  - *Time:* 1 hour

- **Visit St. Patrick’s Cathedral:**
  - *Activity:* Enjoy the beautiful architecture and holiday decorations.
  - *Cost:* Free
  - *Time:* 1 hour

**Afternoon:**

- **Walk down Fifth Avenue:**
  - *Activity:* Window shop and enjoy holiday displays, especially at stores like Macy's (though it may be closed).
  - *Cost:* Free
  - *Time:* 1-2 hours

- **Lunch at Bryant Park:**
  - *Recommendation:* Food kiosks at the Winter Village
  - *Cost:* ~$15
  - *Time:* 1 hour

- **Visit the New York Public Library:**
  - *Activity:* Explore one of the most famous libraries in the world.
  - *Cost:* Free
  - *Time:* 1 hour

**Evening:**

- **Dinner at a cozy restaurant:**
  - *Recommendation:* Keens Steakhouse
  - *Cost:* ~$40
  - *Time:* 1.5 hours
  
- **Walk through Times Square:**
  - *Activity:* Experience the vibrant lights and atmosphere.
  - *Cost:* Free
  - *Time:* 1 hour

---

### Day 3: December 26th

**Morning:**

- **Breakfast nearby your hotel:**
  - *Recommendation:* A local café (e.g., Cafe Zaiya)
  - *Cost:* ~$10
  - *Time:* 1 hour

- **Visit the 9/11 Memorial & Museum:**
  - *Activity:* Pay respect and learn about the events through exhibits.
  - *Cost:* ~$26
  - *Time:* 2-3 hours

**Afternoon:**

- **Lunch at Eataly Downtown:**
  - *Cost:* ~$20
  - *Time:* 1 hour

- **Walk through the Financial District:**
  - *Activity:* See Wall Street, the Charging Bull, and One World Observatory (optional).
  - *Cost:* Free (Observatory costs ~$43)
  - *Time:* 2 hours

**Evening:**

- **Dinner in Chinatown:**
  - *Recommendation:* Xi'an Famous Foods (famous for their hand-pulled noodles)
  - *Cost:* ~$15
  - *Time:* 1 hour

- **Explore Little Italy:**
  - *Activity:* Enjoy the festive atmosphere and holiday lights.
  - *Cost:* Free
  - *Time:* 1 hour

- **Return to your hotel.**

---

### Total Estimated Costs:

- Day 1: ~$15 + ~$15 + $25 + ~$20 + ~$50 = **~$125**
- Day 2: ~$10 + ~$15 + Free + ~$15 + ~$40 = **~$80**
- Day 3: ~$10 + ~$26 + ~$20 + Free = **~$56** (if you skip the Observatory)
  
**Grand Total:** ~$261, leaving you with a comfortable buffer for additional snacks, transport, and any unexpected expenses.

### Transportation:

- **MetroCard for subway:** ~$33 for a 7-day unlimited ride card (great for hopping on the subway).
  
Enjoy your trip to New York City! Happy Holidays!

2. Batch Upload to Vector Store and Query¶

Goal: Add multiple files to a vector store for RAG¶

# Upload files to a vector store.
vector_store_name = "batch_vector_store"
file_paths = [
    "../helpers_root/docs/tools/all.imports_and_packages.how_to_guide.md",
    "../helpers_root/docs/tools/unit_test/all.write_unit_tests.how_to_guide.md",
    "../helpers_root/docs/code_guidelines/all.coding_style.how_to_guide.md",
]  # Example paths

question = "Is `from pathlib import Path` a correct import according to the coding guidelines?"

# Create or find vector store.
llm = hllm.LLMClient(model="gpt-4o")
llm.create_client()
client = llm.client
vector_store = client.vector_stores.create(name=vector_store_name)

# Upload files to the vector store.
file_streams = [open(path, "rb") for path in file_paths]
file_batch = client.vector_stores.file_batches.upload_and_poll(
    vector_store_id=vector_store.id, files=file_streams
)

if file_batch.status != "completed" or file_batch.file_counts.failed > 0:
    raise RuntimeError(
        f"Ingestion not ready: status={file_batch.status}, counts={file_batch.counts}"
    )

resp = client.responses.create(
    model="gpt-4o",
    input=question,
    tools=[
        {
            "type": "file_search",
            "vector_store_ids": [vector_store.id],
        }
    ],
)

# Extract the assistant's text.
out_text = getattr(resp, "output_text", "")

# Best-effort extraction of cited sources from output annotations.
sources: List[Tuple[str, str]] = []
for item in getattr(resp, "output", []) or []:
    for part in getattr(item, "content", []) or []:
        if getattr(part, "type", "") == "output_text":
            annotations = (
                getattr(getattr(part, "text", None), "annotations", []) or []
            )
            for ann in annotations:
                if getattr(ann, "type", "") == "file_citation":
                    file_id = ann.file_citation.file_id
                    fobj = client.files.retrieve(file_id)
                    sources.append((fobj.filename, file_id))


# Display file batch status
print("\n=== ANSWER ===\n", out_text)

if sources:
    print("\n=== SOURCES ===")
    for name, fid in sources:
        print(f"- {name}  ({fid})")

INFO:httpx:HTTP Request: POST https://api.openai.com/v1/vector_stores "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/files "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/files "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/files "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/vector_stores/vs_68c206d0b36c819181486d1ec9b02118/file_batches "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: GET https://api.openai.com/v1/vector_stores/vs_68c206d0b36c819181486d1ec9b02118/file_batches/vsfb_1b51f95157434a8fb9403704afa4f228 "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: GET https://api.openai.com/v1/vector_stores/vs_68c206d0b36c819181486d1ec9b02118/file_batches/vsfb_1b51f95157434a8fb9403704afa4f228 "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: GET https://api.openai.com/v1/vector_stores/vs_68c206d0b36c819181486d1ec9b02118/file_batches/vsfb_1b51f95157434a8fb9403704afa4f228 "HTTP/1.1 200 OK"
INFO:httpx:HTTP Request: POST https://api.openai.com/v1/responses "HTTP/1.1 200 OK"


=== ANSWER ===
 According to the coding guidelines, `from pathlib import Path` is not the recommended import style. The guidelines suggest avoiding `from ... import ...` statements in general. Instead, it is recommended to use imports that start with `import`, which helps with maintenance, avoids potential name collisions, and improves debugging clarity.

3. Apply Prompt to DataFrame¶

Goal: Run prompts batch-wise on a lot of data¶

df = pd.DataFrame(
    {
        "question": [
            "Summarize: Attention is all you need.",
            "Summarize: Diffusion models in 2 sentences.",
            "Summarize: Convnets vs Transformers for vision.",
        ]
    }
)

df_out = hllm.apply_prompt_to_dataframe(
    df=df,
    prompt="Summarize each item in one sentence.",
    model="gpt-4o-mini",
    input_col="question",
    response_col="summary",
    chunk_size=3,
    allow_overwrite=True,
)
print(df_out.head())

Processing chunks:   0%|                                                              | 0/1 [00:00<?, ?it/s]

LLM API call ...

INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
Processing chunks: 100%|██████████████████████████████████████████████████████| 1/1 [00:03<00:00,  3.49s/it]

LLM API call done (3.459 s)
                                          question  \
0            Summarize: Attention is all you need.   
1      Summarize: Diffusion models in 2 sentences.   
2  Summarize: Convnets vs Transformers for vision.   

                                             summary  
0  "Attention is All You Need" introduces the Tra...  
1  Diffusion models are generative models that le...  
2  Convnets (Convolutional Neural Networks) excel...

3. Use the Cost Tracker¶

Goal: Find the cumulative or individual costs of your LLM jobs¶

tracker = hllm.LLMCostTracker()
txt = hllm.get_completion(
    "Say hello in 10 words.",
    system_prompt="You are terse.",
    model="gpt-4o-mini",
    cache_mode="NORMAL",
    temperature=0.1,
    max_tokens=1000,
    print_cost=True,
    cost_tracker=tracker,
)

txt2 = hllm.get_completion(
    "Say hello in 50 words.",
    system_prompt="You are terse.",
    model="gpt-4o-mini",
    cache_mode="NORMAL",
    temperature=0.1,
    max_tokens=1000,
    print_cost=True,
    cost_tracker=tracker,
)

txt3 = hllm.get_completion(
    "Say hello in 70 words.",
    system_prompt="You are terse.",
    model="gpt-4o-mini",
    cache_mode="NORMAL",
    temperature=0.1,
    max_tokens=1000,
    print_cost=True,
    cost_tracker=tracker,
)

print("Custom tracker total: $", tracker.get_current_cost())
tracker.end_logging_costs()

LLM API call ...

INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"

LLM API call done (1.650 s)
cost=$0.000010
LLM API call ...

INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"

LLM API call done (1.112 s)
cost=$0.000036
LLM API call ...

INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"

LLM API call done (2.013 s)
cost=$0.000053
Custom tracker total: $ 9.989999999999999e-05