Nexus Random Initialization Foundry
Dense route: random model → training → F32/F16/Q8/Q4 export. LittleBit route: random binary factors → training → required same-corpus recovery → packed export.
GPU not connectedBuild a custom byte-BPE vocabulary
Train a reversible tokenizer on your own text before creating a model. Every UTF-8 byte is representable. BOS = 256, EOS = 257, learned tokens start at 258. Ordinary content never injects special tokens. Training text stays in this browser.
This imports no weights. A new vocabulary needs original raw text. Changing tokenizer selection never changes an existing model.
Byte vocabulary ready. No custom tokenizer trained.Tokenizer training has a bounded sample; the report records bytes/documents used and its hash. Changing tokenizer selection affects the next model only. Existing models retain their exact vocabulary. New vocabularies use the standard SmolLM byte-BPE pretokenizer. Legacy whole-document tokenizers keep their original runtime requirement. The training report counts pretokenized pieces, with a bounded sample.
Create a dense or native-factor model
Intended purpose: a reasoning-focused NexusAI foundation model. A fresh random model has no demonstrated reasoning ability. Perplexity is not a reasoning-accuracy score; capability remains unmeasured until separate task evaluation.
Training sequence default: 1024. Model context default: 2048. Larger attention graphs remain subject to memory/binding limits. Saved runs keep their own settings.
No model yet
One dataset. Your token budget.
TXT: choose the shared document format before creating or importing a tokenizer. Blank-line mode separates documents with blank lines. TinyStories mode splits only at <|endoftext|>, preserving paragraph breaks inside each story and removing the structural separator. Both tokenizer learning and corpus preparation use the same complete stories for deterministic holdout. Parser changes require a fresh page once a tokenizer/model/session is bound. TinyStories admits at most 4,000,000 source records; ordinary tokenizer sampling remains capped at 1,000,000. Input admission automatically follows the selected source, up to a 3072 MiB hard limit. Gzip expansion is unknown until streamed and remains bounded by that limit. A manual cap is available in Advanced settings. Training uses one end-of-document EOS; BOS follows the imported tokenizer flag (always enabled for Foundry bytes). JSONL: one object per line with text, content, code, or messages. JSONL and TXT are streamed into bounded token buffers. JSON arrays are limited to 48 MiB. Explicit train/dev/test labels are respected; document families cannot cross splits.
Curriculum: new material + reinforcing replay
Disk-backed corpora retain original document order and do not support curriculum or queue replay. Training, held-out evaluation, checkpoints and same-corpus recovery read bounded pages from disk.
Supports the same TXT/JSONL/JSON imports above. Original training buffers are preserved. Dev/test documents never enter the feeding plan. Each later curriculum pass mixes newly introduced training material with older material from earlier passes in the same ingestion stream. This replay reinforces exposure; it is not a separate retraining job, reinforcement learning or a teacher. New means first exposure within this plan, not necessarily unknown to the model across its lifetime. Native LittleBit recovery is a separate optimization phase; it can use the same feeding plan.
Prepare a corpus first.
Whole documents remain intact, so realized token fractions can differ from requested fractions. First-pass replay is unavailable; shortages and overshoot are reported rather than silently substituted. The training goal becomes the exact realized plan length. Checkpoints embed the plan and reapply it after the original corpus is imported.
No feeding plan active.
Queue up to10 next corpus levels
Each selected group is one next corpus. The current corpus finishes first, including required native recovery. A full saved-state perplexity and selected academic gate must finish before learned weights move to the next level. The tokenizer stays frozen; each next level gets fresh Adam, schedule and progress.
Advanced queue feeding
Remaining targets replay earlier training-only material in the same stream. Current and historical held-out rows are excluded. The plan records unique and repeated exposures. All selected token buffers must fit the host-memory ceiling.
Queue checkpoint and file reattachment
Full checkpoints contain queue position and feeding policy. After reload, restore that checkpoint and reattach every original file group in order. A manifest alone is not a model checkpoint.
No automatic queue configured.
Next-token model training
Binary U/V latent masters and h/g/l scales learn from next-token loss in every matrix. Initialization needs no dense teacher, SVD or compression pass. A same-corpus recovery pass follows the primary training budget. AdamW, clipping, warmup and cosine decay remain available.
Automatic checkpoint + perplexity failsafe
Choose when to save a complete paused-state checkpoint and test its exact held-out perplexity. Choose a writable folder first. Downloads and RAM snapshots are not verified disk checkpoints.
Save or evaluation failure always pauses. The latest and best checkpoint are tracked separately. No automatic restart after rollback. Closing the page still requires an explicit restore.
No automatic checkpoint yet.
Optional RAM error watchdog
Timing starts with the next training session.
Rates count targets only after an optimizer update completes. Recent speed is smoothed over approximately 3 seconds; displays refresh at most every 350 ms during training. ETA covers this stage only and excludes pauses, evaluation and checkpoint saving.
Restore full training state
After restoring, prepare the identical corpus before restoring its cursor. GGUF stores signed factor bits and scales. .nxnative also stores the real-valued latent masters, Adam moments and progress needed for exact training continuation. Checkpoints are explicit; save before closing the tab.
Activity log
Export the representation you trained
Export measurements
Portable llama GGUF
Materialize the current forward weights into standard dense tensors. For native factor models this file is larger than the compact checkpoint. Save a full checkpoint separately to resume training.
No export yet.
Check what the model learned
Explicit dev documents are excluded from training. Unlabelled corpora use a deterministic 10% document-family holdout. Test/probe documents are excluded from both optimization and dev evaluation.
Loading the last export replaces the working model and clears its prepared corpus. Save a checkpoint first. Reimport the same corpus to compare pre/post-compression loss. Native GGUF loading preserves the factor representation. The adapted native recovery pass is integrated above; the original v14 tool below remains available for its SmolLM2-135M workflows.
Numerical tests are available on your actual GPU.
GSM8K · ARC-Challenge · BIG-Bench Hard
Use the supplied official-data pack. These are real evaluation records, held out from training. A fixed subset is not a full benchmark result. Nexus zero-shot direct-answer scoring differs from published leaderboard protocols. Every score is bound to the exact saved checkpoint, tokenizer, record IDs and decoding settings.
Preflight checks every selected prompt plus its output budget against the model context before training starts. No silent truncation or easy-question substitution. With short contexts, select an explicitly labeled filtered subset or change the configuration. Generated answers reaching the fixed output budget are scored under that declared budget and marked truncated; invalid answers count incorrect. Real inference errors, missing records or partial suites pause the gate.
No academic model scores yet. Load the pack, then preflight the selection.
Recover on the same corpus before export
Adapted from v14’s recovery-cycle workflow. The app requires this phase for native 1/0.5/0.25-bit cycles, using the exact training corpus and tokenizer. Use additional training on the prepared dataset, or add a compatible native teacher’s next-token probabilities. This is supervised training / distillation, not reward-based reinforcement learning.
Advanced teacher assistance · off by default
Scan the selected model to see which interfaces this runtime actually supports. Your NexusAI tokenizer stays independent. A scan never installs model code or implements an unknown architecture.
Teacher assistance is off.
Direct logit teaching supports compatible Foundry dense/native GGUFs with the exact tokenizer. Arbitrary Mistral/other architectures or different vocabularies are rejected. Teacher-generated text from another model can be imported as ordinary training data; it is not live logit distillation.
One existing token budget: default 2× for the first 10%, then 1× at the exact valid-target boundary; 2% token warmup and cosine. Full masters/Adam/cursor snapshot, baseline + first-update + boundary held-out gates. On rejection restore verified snapshot and pause; next Resume uses1×. Does not add corpus passes. Maximum effective LR0.001. Strong-phase rollback payload is included in full checkpoints.
No two-phase recovery yet.
Advanced direct teacher file
No teacher loaded
Teacher and student must share the exact tokenizer fingerprint. A better-trained native model can teach the student; an untrained teacher is not useful. The teacher remains frozen. Extra GPU memory is required.No recovery pass yet.
Pretraining optimizer moments are retained. The pass gets a fresh token-budget learning-rate schedule. v14’s original SmolLM2-135M Q8-teacher / hidden-state recovery and masked job-SFT remain available below; their pinned tokenizer and model restrictions still apply.
Advanced decoded-text teacher interface
TEACHER TEXT COMPATIBILITY BRIDGETeacher text → your own student tokens
Keep your custom NexusAI tokenizer. A supported teacher uses its own tokenizer, generates text, and the bridge re-encodes that text with the student tokenizer. This is an explicit sample-generation/export/import workflow, not cross-vocabulary KL or automatic live teaching. Mistral14B/arbitrary architectures are not supported by this loader. Decoded teaching text from any external model can be imported through ordinary TXT/JSONL.
No bridge teacher loaded
Student tokenizer and weights remain unchanged during sample generation. Import exported samples with a separate dev corpus to train on them.
Optional original tools
The lightweight build keeps the repaired Foundry separate from the large original tool payloads. Your unchanged original Foundry still contains Adapter Studio, Glass Chat and v14 Recovery. You can also open an original tool HTML below. Its model state is separate and its original architecture/tokenizer restrictions still apply. Use the main Foundry to test new byte-token models; use the original tools for their supported SmolLM2 workflows. Export a supported model, then select that file inside the tool. Closing the tool releases its frame; save its work first.
Complete one corpus cycle at a time
NexusAI is the model/application identity. The tokenizer and all weights can be created from scratch. Architecture sizes describe parameter counts, not demonstrated intelligence, production readiness, or feasible billion-parameter browser training.
Grok reference adaptation and provenance
The supplied nexus_corpus_training_code.txt is a PyTorch continuation/recovery reference. Its batch 16 × sequence 256, CE-only default, AdamW β₁=.9 / β₂=.95, weight decay 0, warmup 100 optimizer updates, cosine floor .1 and clipping 1 are implemented as selectable browser settings. Factor-specific LR groups and SmoothSign are available only for the native-factor backend. CPU threads=8, fused PyTorch kernels and BF16 are not implemented in this browser. The original reference is preserved in the source package; it was inspected, not executed. Choosing these settings does not establish optimal TPS on your GPU. LittleBit attribution and original upstream licences remain separate from the NexusAI branding.
- Run the native GPU self-test. Start with a small 0.5-bit run; use the 7.3M or 99.5M architecture for a 0.25-bit target.
- Create a native factor model. Select Nexus Core Corpus and import the recombined Nexus_Core_Corpus.jsonl. Train this core stage first; broader English data can follow with core examples retained in the mixture. Dataset order does not guarantee identity or learned behavior.
- Set 10,000 training tokens and start. Check held-out loss and save a full checkpoint.
- For LittleBit, complete the automatically scheduled recovery pass before exporting native GGUF; dense models can export after training. Scale the architecture and token budget after validation.
Native factor pretraining can take substantial time. Keep this tab open and prevent system sleep. This build does not claim a particular Radeon utilization, frontier capability, or quality retention at 0.25 bits. New models have an adjustable 8–512-token context ceiling. Training is FP32; Q4 is a post-training dense export target, not native 4-bit arithmetic. Native factor precision is a packed binary-factor storage target, not 4-bit quantization-aware training. Dataset fetch requires internet access and a working dataset viewer; local imports and training work offline.
Provenance, references and validation scope
Uses the supplied Nexus SmolLM2 Adapter Studio’s tokenizer, GGUF decoding, WebGPU kernels, autograd, AdamW and transformer forward graph. New work replaces post-training factorization with randomly initialized, natively trained binary factors for all matrices; adds clipped STE differentiation, FP16 scale rounding, direct factor export and latent-master checkpoints. Dataset ingestion and exact-budget sessions are retained. Original Glass and v14 tools remain preserved in the unchanged source upload and optional full build.
LittleBit research credit: Banseok Lee, Dongkyu Kim, Youngcheon You and Youngmin Kim. This is an independent experimental adaptation, not an official Samsung implementation. It retains your packed U/V + h/g/l representation. The supplied recovery code used tanh(100x) surrogate gradients and a Q8 teacher; this native path uses clipped identity STE and corpus next-token loss. The native path uses no Dual-SVID initialization. Its required recovery is corpus CE with optional teacher-logit KL; v14’s original five-stage hidden-state recovery remains restricted to SmolLM2-135M.
This upgraded build requires new regression verification. Prior-build evidence below does not certify the new batching/tokenizer paths. This repair includes actual-source regression tests, native SwiftShader WebGPU numerical tests, and independent dense GGUF loading/inference in llama.cpp. Training fixture results do not establish general model capability. The browser interface and physical Radeon execution have not been tested in this environment. Run the native GPU self-test on your machine before a long training run. Native-factor GGUF uses a custom architecture and requires the Foundry runtime; standard F32/F16/Q8_0/Q4_0 exports use dense Llama tensors.