Building a Cloud-Native Agentic AI Research App with pgvector and Multimodal LLMs

Transitioning from traditional on-premises servers to containerized, cloud-native microservices represents the single most significant shift in software engineering over the last decade. Back in 2014, I built a Three-Tier client-server application for image retrieval using C++ and Node.js. It relied on SURF descriptors and CIELAB color space clustering—cutting-edge for its time, but monolithic and brittle by today's standards. Recently, I replatformed this academic research into a modern Cloud-Native architecture on AWS, evolving it into the Agentic Research App.

In this guide, we will explore the implementation of a multimodal Retrieval-Augmented Generation (RAG) pipeline. To achieve production-grade performance, developers often struggle with API latency and model reliability. This is where n1n.ai provides a critical advantage by offering a unified, high-speed gateway to top-tier models like OpenAI o3, DeepSeek-V3, and Claude 3.5 Sonnet. Using a stable aggregator like n1n.ai ensures your agentic workflows remain resilient even during provider outages.

The Architectural Stack

To handle intensive AI processing while maintaining a snappy user experience, the architecture is divided into specialized layers:

1. Frontend & API Gateway: React and Remix (Vite), styled with Tailwind CSS and Shadcn UI.
2. Backend Logic: Node.js, built on the @greeneyesai/api-utils MVC framework.
3. AI Integration: Multimodal capabilities via Gemini 2.5 Flash and GPT-4o. For developers looking to scale, n1n.ai offers a seamless way to switch between these models via a single API key.
4. Database: PostgreSQL 16 with the pgvector extension for vector similarity search.
5. Cache & Pub/Sub: Redis for rate-limiting and real-time streaming via Server-Sent Events (SSE).

Deep Dive: Vector Similarity with pgvector

Instead of specialized SaaS vector databases, I chose pgvector to keep the architecture consolidated. This reduces network overhead and simplifies relational joins. Below is the Dockerfile configuration to ensure compatibility with Alpine Linux:

FROM postgres:16-alpine3.19

# Install build tools for pgvector
RUN apk add --no-cache git g++ make musl-dev postgresql-dev

# Install pgvector v0.6.2
RUN git clone --branch v0.6.2 https://github.com/pgvector/pgvector.git \
    && cd pgvector \
    && make \
    && make install \
    && cd .. \
    && rm -rf pgvector

Once the database is up, we initialize the schema with an ivfflat index. This index type is optimized for speed by partitioning the vector space into clusters. For high-dimensional embeddings (e.g., text-embedding-3-large), we use the cosine distance operator (<->):

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE memory (
    id SERIAL PRIMARY KEY,
    user_id BIGINT NOT NULL,
    content TEXT NOT NULL,
    embedding VECTOR(768),
    created_at TIMESTAMP DEFAULT NOW()
);

CREATE INDEX memory_embedding_idx ON memory
USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100);

Implementing the Multimodal RAG Pipeline

In the 2014 version, extracting text required a heavy C++ CLI tool using Tesseract OCR. Today, we utilize multimodal LLMs like GPT-4o or Claude 3.5 Sonnet. When a user uploads a PDF or image, the backend processes the buffer and sends it to the LLM for extraction.

Pro Tip: When building for scale, use DeepSeek-V3 for cost-effective extraction and OpenAI o3 for complex reasoning. Using a provider like n1n.ai allows you to route these requests dynamically based on the file type and priority.

async function extractFileText(file: Express.Multer.File): Promise<string> {
  const base64Data = file.buffer.toString('base64')
  // Request to LLM via n1n.ai aggregator
  const result = await llmClient.generateContent([
    'Extract all readable text. Output only raw text.',
    { inlineData: { data: base64Data, mimeType: file.mimetype } },
  ])
  return result.text
}

Real-Time Streaming with Redis and SSE

Waiting for a full LLM response creates a poor UX. We implement streaming using Redis Pub/Sub. The backend publishes chunks to a Redis channel, and a dedicated SSE endpoint streams them to the client.

In the Node.js controller, we handle the subscription:

public async eventSourceForUser(req: Request, res: Response): Promise<void> {
  const { userId } = req.params;
  res.writeHead(200, {
    "Content-Type": "text/event-stream",
    "Cache-Control": "no-cache",
    "Connection": "keep-alive",
  });

  this.redis.subscribe(`CHANNEL_${userId}`, (chunk: string) => {
    res.write(`data: ${chunk}\n\n`);
  });

  req.on("close", () => this.redis.unsubscribe(`CHANNEL_${userId}`));
}

Agentic Personalities and Tool Use

An agent is only as good as its context. By using the Strategy Pattern, we inject different "personalities" into the system. For instance, a "Research Assistant" personality might emphasize citations, while a "Creative Writer" focuses on flow.

We also integrate tools like the Bing Search API. If the vector search in pgvector doesn't yield high-confidence results, the agent triggers a web search to augment its memory. This "Agentic Loop" ensures the LLM doesn't hallucinate when its internal training data or provided context is insufficient.

Production Deployment on AWS Fargate

Deploying a stateful AI app requires robust orchestration. We use AWS Fargate for the API containers to avoid managing EC2 instances.

• Auto-scaling: We scale from 2 to 16 instances based on CPU utilization during heavy LLM processing.
• Database: Amazon Aurora PostgreSQL provides the high availability needed for pgvector operations.
• Networking: An Application Load Balancer (ALB) handles SSL termination and sticky sessions for SSE.

Conclusion: The Future of Agentic Systems

Modernizing legacy systems into Cloud-Native AI agents is no longer a luxury—it is a necessity for staying competitive. By leveraging pgvector for memory, Redis for real-time interaction, and AWS for scale, we can build research assistants that truly understand and synthesize information.

For developers starting this journey, managing multiple API keys and monitoring usage across different providers is the biggest bottleneck. I highly recommend using a unified LLM API aggregator to streamline your development and ensure 99.9% uptime.

Get a free API key at n1n.ai.