Skip to main content

Microservices API Documentation

Microservices API​

Welcome to the RiverGen Microservices API documentation. This documentation covers all internal microservices that power the RiverGen platform.

Microservices provide specialized functionality that is separated from the main API for scalability, maintainability, and independent deployment. Each service has its own API endpoints and can be scaled independently.

Quick Navigation

Architecture​

RiverGen's microservices architecture consists of:

  1. Data Orchestration Service - Query execution, catalog management, and data lineage
  2. Storage Service - File storage, quota management, and model registry
  3. Staging Service - Data staging for Model Studio consumption

These services communicate with the main API through HTTP requests and share the same database and infrastructure.

Available Services​

Data Orchestration Service​

Base URL: http://data-orchestration:8000 (internal) or http://localhost:8000 (development)

Purpose: Handles query execution, data catalog management, schema discovery, and data lineage tracking.

Endpoints: 17 endpoints

  • Health & Status (2)
  • Execution (2)
  • Connectors (3)
  • Catalog (7)
  • Lineage (3)

Documentation: Data Orchestration Service

Storage Service​

Base URL: http://storage-service:8002 (internal) or http://localhost:8002 (development)

Purpose: Manages file storage, quota tracking, vector storage, caching, and model artifacts.

Endpoints: 22 endpoints

  • Files - Public API (5)
  • Quota - Public API (2)
  • Vectors - Internal API (3)
  • Files - Internal API (2)
  • Cache - Internal API (3)
  • Lifecycle - Internal API (3)
  • Model Registry - Public API (4)

Documentation: Storage Service

Staging Service​

Base URL: http://staging-service:8003 (internal) or http://localhost:8003 (development)

Purpose: Handles data staging operations for Model Studio, converting data sources to Parquet format for ML consumption.

Endpoints: 8 endpoints (Not Implemented)

  • Job Management (4)
  • Job Control (2)
  • Chunk Management (2)

Status: All endpoints return 501 Not Implemented

Documentation: Staging Service

Quick Start​

  1. Understand Architecture: Review the Microservices Overview to understand the architecture
  2. Data Operations: Use Data Orchestration Service for query execution and catalog management
  3. File Management: Use Storage Service for file operations and model artifacts
  4. Data Staging: Check Staging Service for Model Studio integration (coming soon)

Base URLs​

ServiceInternal URLDevelopment URLPort
Data Orchestrationhttp://data-orchestration:8000http://localhost:80008000
Storage Servicehttp://storage-service:8002http://localhost:80028002
Staging Servicehttp://staging-service:8003http://localhost:80038003

Authentication​

Microservices use header-based authentication:

X-Org-ID: <organization_id>
X-User-ID: <user_id> (optional)

Note: Most microservices are called internally by the main API, which handles authentication and passes the organization context via headers.

Response Format​

All API responses follow a consistent structure:

{
  "success": true,
  "data": { ... },
  "message": "Operation successful"
}

Error responses:

{
  "success": false,
  "error": {
    "code": "ERROR_CODE",
    "message": "Human-readable error message"
  }
}

Status Codes​

  • 200 - Success
  • 201 - Created
  • 204 - No Content
  • 400 - Bad Request
  • 401 - Unauthorized
  • 403 - Forbidden
  • 404 - Not Found
  • 409 - Conflict
  • 422 - Validation Error
  • 429 - Too Many Requests
  • 500 - Internal Server Error
  • 501 - Not Implemented

Internal Notes​

  • Microservices are typically accessed internally by the main API
  • Direct access to microservices requires proper network configuration
  • All services share the same PostgreSQL database
  • Services use header-based authentication (X-Org-ID, X-User-ID)
  • Internal APIs (marked as "Internal") are for service-to-service communication only
  • Public APIs can be accessed directly but require proper authentication headers