Computational GRID and Metacomputers

Allow 90-day pilot on three autonomous circuits

To check whether it is possible to assemble a temporary metacomputer for the task, perform a calculation next to the data and release the reproduced result - without centralizing the primary arrays.

Main thesis
Don't build another supercomputer. Create a trusted layer that combines disparate power, data, and rules into a managed metacomputer for the duration of a specific task.

Recommended solution
Approve the design pilot; industrial scaling, real sensitive data and inter-circuit records allow only individual solutions.

03 / PROBLEM AND PURPOSE EFFECT

There are capacities, but they do not form a common managed resource.

Clusters, clouds, accelerators and databases belong to different organizations, use different queues, policies, identifiers and result formats.

• Break 1. It is difficult to know where CPU, GPU are, memory, network, licenses, and local data.
• Break 2. Planners manage the site well, but do not see the powers and capabilities of neighboring domains.
• Break 3. Copying large arrays to the center creates delays, takes, leak risk, and owner dispute.
• Break 4. Closely related HPC tasks, mass ensembles, and AI loads require different executive circuits.
• Break 5. A result without a code version, inputs, policies, and environment cannot be independently reproduced.
• Break 6. Cost, energy, waiting in line, and network egress are often considered separate.

Target effect
Reduce the path from a computational query to a verifiable result, while maintaining local sovereignty of sites and data owners.

04 / TERMS MAP

Four definitions define the limits of the system

Criteria GRID are based on the work of Ian Foster: coordination of resources of different administrative domains, open universal protocols and non-trivial quality of service [1–2].

Formula
GRID = TRUST + CATALOGUE + PRIVACY + MARCHRUTIZATION + EXECUTION + TRAINING + PROOF

05 / PRINCIPLES AND INVARIANTS

A federation is strong only with predefined restrictions.

1. Local Sovereignty. The site retains the owner, local planner, quotas, access rules, and the right to refuse.

2. If possible, the code, plan, and allowed projection are moved to the data, not the entire source array.

3. The general layer sees only the required resource card, policy, state and agreed result.

4. Open contracts. canonical descriptions of the task, resource, data and result are separated from the specific product.

5. Different Load Routes. MPI/HPC, HTC, AI/GPU, SQL- federation and streaming tasks are not reduced to one queue.

6. Policy like code tolerance, data zone, deadline, target, egress limit and authority are checked before and during launch.

7. Reproducibility: The result contains versions of code, environment, inputs, parameters, policies, and checksums.

8. Independent verification. the operator does not confirm its own safety and correctness alone.

Invariant
The local outline can reject the task; the general layer does not bypass local policy and does not require a single root access.

06 / PLACE IN ECOSYSTEM

GRID turns an analytical query into a verifiable computational execution

Formula
QUESTION → PASSPORT OF THE TASK → METACOMPUTER → CALCULATION → VERIFICATION → ARCHIVE → NEW CYCLE

07 / TASK ARCHITECTURE

Five planes share powers and flows

1. Trust. Federal identification, attributes, roles, certificates, service accounts, review and decision log.

2. Management. resource catalog, job broker, policy engine, orchestrator, quotas, SLA and adapters to local systems.

3. Calculations. SlurmMPI, HTCondor/workflow, Kubernetes Jobs, GPU/FPGA and other specialized executive contours.

4. Data. domain databases, object and file storages, directories, connectors, pushdown, staging and release projections.

5. Evidence. telemetry, accounting, lineage, checksums, manifestos, independent verification and Heritage Archive.

Separation of flows
The control layer transmits tasks and state; mass data goes through the permitted data path directly between sites or is processed on-site.

Open interfaces reduce vendor binding; local products are connected by adapters [3–4].

08 / CONTOUR OF CONFIDENCE AND MANAGEMENT

Before launch, the system should answer seven questions

The Zero Trust
Network location is not a sufficient basis of trust. Access is assessed for a specific subject, resource and context, and the decision is enforced at the point of application of the policy [24].

09 / METACOMPUTERS FORMATION

The metacomputer is collected for the duration of the task and then disassembled

1. The task passport fixes the target, code, data, SLA, limitations, error price and result criteria.
2. The directory returns compatible sites, software versions, accelerators, network conditions, and data locality.
3. The broker builds several execution plans and evaluates queue time, data portability, cost, and risk.
4. Policy engine checks each site, data set, image, authority and valid result.
5. The orchestrator reserves resources and translates the canonical assignment into Slurm, HTCondor or Kubernetes.
6. During execution, telemetry, checkpoints, policy events, resource accounting, and lineage are collected.
7. After verification, the result is issued; temporary credentials are withdrawn, and the staging is cleared by policy.

Limitation
WAN does not turn into a local bus. A closely related MPI problem is usually placed entirely in one low-latency HPC circuit; tasks, data, and results are federalized between sites [10].

10 / LOADING AND MARCHRUTIZATION CLASSES

The right backend is more important than a single universal queue

Slurm manages resources and queues within the cluster [5–7]; HTCondor optimizes the accumulated amount of calculations over time8–9]; Kubernetes Jobs is one of the backends for completed container tasks [11–13].

11 / FEDERAL DATABASE

The request passes between the contours, the primary data remains with the owner

Catalog. publishes Dataset ID, owner, schema, version, quality, classification, statistics and endpoint without the mandatory publication of the data itself.

Semantics. domain dictionaries and versioned mappings link terms; a single giant scheme is not required.

The request plan. filter, projection, aggregation and allowable join are transferred to the source; the transfer plan and evaluation are checked before launch.

Data path. Arrow Flight/Flight SQL or other gateways transmit allowed large flows; transport does not replace authorization.

Materialization. Agreed derivative sets and results are recorded as versioned snapshots; the sources remain domain-specific.

Record. interdomain OLTP is not considered atomic without a proven distributed commit; local record, idempotent jobs and sagas are preferred.

Rule of admission
Full scan, cross-source cross join and unlimited egress are prohibited by default. First, pushdown, selectivity, result limit and allowable cost [15–19] are proved.

12 / DATA CONTRACT AND ORIGIN

Each entry and result receives a passport, not just a file name

Heritage archive
Stores the manifest, versions, access decisions, evidence, and permitted results. Sensitive sources may remain with the owner; the archive retains the reference and origin [20–22].

13 / LIFE CYCLE OF THE TASK

Eight gateways make the launch manageable and controversial

Fail closed
If there is no mandatory attribute, policy, statistics, signature, validator or exit limit, the task does not move to the next gateway.

14 / SOVEREIGNTY AND SAFETY

Protection is built around the resource, data and output of the result

Identity. Short-lived service credentials, mTLS, trust federation, attributes and quick feedback.

Authorization. policy decision for each call and task; the local point of application does not trust the central solution without verification.

Isolation. separation of tenant/job, sandbox, network policy, protected secrets, signed images, and allowlist artifacts.

Data. classification, minimization, masking, encryption, keys in the local outline and egress control.

Result. check for disclosure, aggregation, line/volume limits, labeling, owner approval.

Supply chain. source code version, SBOM, build signature, scan, provenance and playable image.

Incident. task isolation, credentials recall, preservation of evidence, owner notification and independent review.

Russian Outline
Prior to the pilot, a legal classification of data and systems is required 149-FZ, 152-FZ, 187-FZ and by-laws. This document is not a legal opinion[25–27].

15 / RELIABILITY AND SAFETY

A single domain failure should not become a federation failure.

Formula
RELIABILITY = IDEMPOTENCE + CHECKPOINT + LOCALIZATION OF THE CANCEL + OBSERVATION + VERIFICATION

16 / OBSERVATION AND RESOURCE ACCOUNTING

A single trace links the question, plan, launch, data, and result

Unit of calculation
The account refers to a specific job/run and is confirmed by the local site. The proposed pilot tolerance for the discrepancy between consolidated and local accounting is not more than ±5%.

OpenTelemetry links trades, metrics, and logs through a common context [23]; Slurm supports task and step resource accounting [7].

17 / PILOT TOPOLOGY

Three domains are enough to test the principle of federation

• Total layer. Federated identity, catalogs, policy engine, broker, orchestrator, telemetry and lineage.
• Local layer. Owners retain planners, data, keys, quotas, logs, and right STOP.
• independent control. Separately checks safety, measurements, reproducibility and GO criteria.

Not included automatically
Critical CII, combat personal data, intercontour OLTP and promise of production SLA. Their connection requires a separate admission.

18 / THREE Scenarios

Pilot checks different computational modes with one rule system

Baseline
Prior to integration, each scenario is executed in an existing way. The comparison is built with a fixed base in time, manual labor, bytes moved, cost and quality of the result.

Scenarios are a proposal for approbation, not a statement of performance achieved.

19 / PLAN ON 90 DAYS

Five control stages lead from passport to independent solution

Change management
Any extension of data, sites, powers or public promises is issued as a new version of the perimeter and is re-allowed.

20 / KPI AND NOTE CRITERIA

GO requires both functional, safe and economic results

All numerical thresholds are the pilot’s design criteria. They are not an industry standard and are adjusted prior to launch along with the baseline.

21 / RISKS AND MANAGEMENT MODEL

The main risk is to create the appearance of unity where clear boundaries are needed.

STOP-criteria
Unresolved egress, circumvention of local politics, loss of origin, critical vulnerability without compensation or the inability to independently verify the pilot.

22 / SOLUTION AND NEXT

GRID creates a common order of trusted computing, not a common store of resources

Final thesis
The site stores resources, data and rules. GRID adds overall identity, catalog, planning, evidence, and safe release of the result.

GO. all critical criteria are met; the next queue with new nodes and loads is allowed.

CONDITIONAL GO. value is confirmed, but there is a limited list of closing gaps and a deadline for rechecking.

STOP. sovereignty, security, reproducibility or economics are not confirmed; the prototype is archived without industrial expansion.

Requested action
Appoint a pilot owner, three domain owners and an independent validator; approve a passport and baseline within 15 days.

23 / OFFICIAL AND PRIMARY SOURCES · 1

GRID, planning and executive contours

[1] Ian Foster: What Is the Grid? A Three Point Checklist

[2] Foster, Kesselman, Tuecke: The Anatomy of the Grid

[3] Open Grid Forum: Standards and interoperability

[4] Open Grid Forum: Authorization Framework (GFD.38)

[5] Slurm Workload Manager: Overview

[6] Slurm Workload Manager: Federation Guide

[7] Slurm Workload Manager: Accounting and Resource Limits

[8] HTCondor: High-Throughput Computing and its Requirements

[9] HTCondor Manual: Job Scheduling

[10] MPI Forum: MPI 5.0 Standard

[11] Kubernetes: Jobs

[12] Kubernetes: Scheduling Framework

[13] Kubernetes: Device Plugins

[14] Globus: Collections and Endpoints

Date of treatment: 29 August 2026 Versions of software products and documents should be re-checked before pilot design.

24 / OFFICIAL AND PRIMARY SOURCES · 2

Federated data, origin, observation and safety

[15] Trino: Concepts and federated catalogs

[16] Trino PostgreSQL connector: Pushdown

[17] PostgreSQL: postgres_fdw Remote Query Optimization

[18] Apache Iceberg: Table Specification

[19] Apache Arrow: Flight RPC and Flight SQL

[20] W3C: Data Catalog Vocabulary DCAT 3

[21] W3C: PROV-O — provenance ontology

[22] OpenLineage: Object Model

[23] OpenTelemetry: Signals and context propagation

[24] NIST SP 800-207: Zero Trust Architecture

[25Federal Law No 149-FZ "On Information, Information Technology and Information Protection"

[26Federal Law No 152-FZ "About personal data"

[27Federal Law No 187-FZ "On the security of critical information infrastructure of the Russian Federation"

Legal references are given for guidance. Applicability of norms, editorial and regulatory requirements is confirmed by specialized lawyers and owners of information systems before data access.