Skip to content

Dataset Exports with PostgreSQL

In this example, we'll export large datasets (100K+ rows) from PostgreSQL using structured Pydantic models for field validation and data consistency. Database exports bypass LLM token limits for massive data retrieval.

1. Export your OpenAI API key.

export OPENAI_API_KEY=<YOUR_OPENAI_API_KEY>

2. Define your data model and export structured datasets.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
from toolfront import Database
from pydantic import BaseModel, Field

class Customer(BaseModel):
    customer_id: int = Field(..., description="Unique customer identifier")
    name: str = Field(..., description="Customer full name")
    email: str = Field(..., description="Customer email address")
    total_orders: int = Field(..., ge=0, description="Number of orders placed")
    lifetime_value: float = Field(..., ge=0, description="Total customer spend")

# Connect to PostgreSQL database
db = Database("postgresql://user:pass@host/db", model="openai:gpt-4o")

# Export structured dataset with field validation
customers: db.Table[Customer] = db.ask(
    "Get all customer data with ID, name, email, order count, and total spent"
)

# Process the exported data
df = customers.to_dataframe()
high_value = df[df['lifetime_value'] > 5000]

print(f"Exported {len(customers):,} customers")
print(f"High-value customers: {len(high_value):,}")
print(f"Average LTV: ${df['lifetime_value'].mean():.2f}")

>>> Exported 125,847 customers
>>> High-value customers: 3,421
>>> Average LTV: $1,247.83

Dataset Exports

Use db.Table[Model] to export massive datasets with Pydantic validation. Data is streamed efficiently without LLM token costs, perfect for ETL workflows.