FreeToken serves a 35B model on 8GB VRAM
UC Berkeley’s FreeToken is the day’s sharpest deployment result: MoE routing pushes a 35B model onto an 8GB GPU and reports 39.3 tokens/s, directly widening the floor for local inference.
Technical AI signals · Daily at 8 PM ET
Saturday, August 22, 2026
Today centered on concrete operator artifacts: Berkeley’s FreeToken squeezing a 35B model onto an 8GB GPU, SemiAnalysis putting inference efficiency in tok/s per megawatt, and Microsoft introducing a backend-state benchmark for policy-constrained agents.
UC Berkeley’s FreeToken is the day’s sharpest deployment result: MoE routing pushes a 35B model onto an 8GB GPU and reports 39.3 tokens/s, directly widening the floor for local inference.
The useful new artifact is the metric itself: tok/s per provisioned megawatt gives teams a cleaner way to compare serving stacks, with a cited 7,368 output tok/s/MW figure for a B300 DeepSeek V4 setup.
Thinkingbox moves agent eval toward operational reality with 507 policy-conditioned workflows across five industries and scoring based on actual backend state changes, not just text judgments.
This is a concrete local-serving recipe, not a vague claim: a 32GB Arc Pro B70 running GPTQ INT4, vLLM XPU, FP8 KV cache, and MTP2 speculative decoding hit 52.2 tokens/s on Qwen3.8-27B.
TMI points at a practical path from messy user traces to reusable agent structure, separating multi-threaded recordings into latent tasks with 0.974 agreement against ground-truth groupings.
The build adds automatic context compaction, local files, terminal access, computer use, and a Chrome extension that drives ChatGPT-side actions.
The repo is described as open-source, free, and set up in two minutes to show 90-day trends of productivity with agents.
The project creates persistent AI bots with memory, browser, terminal, and computer access, and can run locally with a user-chosen model.
The thread explains that LLM inference is often bottlenecked by many small kernel launches and says CUDA graphs can nearly double inference speed by reducing CPU chatter.
This release upgrades to OpenAI Python 3.x, switches from httpx to httpx2, and adds --key support to llm embed and llm embed-multi.
The post argues that coding agents need clear instructions and verification, not just line-by-line review.
Linus Torvalds says an AI handled much of a debug session's grunt work, though the AI repeatedly said the bug was impossible and unsolvable.
The preregistration asks whether quantization changes welfare-relevant indicators in open-weight language models, including valence and stability.
The post argues that large heterogeneous groups are incentivized to collaborate on increasingly general shared sub-goals.
The post compares genome architecture and neural-network inductive bias as mechanisms that shape variation along repeated environmental directions.
Commenters say Rust Glancer is a lower-memory Rust language server and note that LLMs are good at building LSP servers.
Commenters discuss remote MCP servers becoming ordinary HTTP workloads and question how many servers will implement the new authorization flow.
Commenters describe Munder Difflin as a local multi-agent harness that wraps existing Claude Code and Codex subscriptions.
Commenters point out related names like 23Labs and 41labs, and one author says a related web server had to be brought back up.
Commenters split on TUIs, with some arguing keyboard-first interfaces are faster and others calling for more TUI apps.
Commenters say OpenTelemetry SDKs are hard to use, especially around automatic instrumentation and stateful abstractions.
Commenters read the extension as adding private repos or general server-side non-public data to ATProto.
Commenters cite manufacturing variability as a challenge for analog computing and ask for clearer LLM benchmarking details.
Commenters discuss superoptimization, web-request latency, and a claim that stochastic search can only maintain or improve an executable objective.
Commenters say Autolith combines a programming agent with a live runtime, and one commenter compares the idea to actor/mailbox systems in Smalltalk, Erlang, or Elixir.
Commenters say Hister builds a personal search index from pages you save and adds semantic search for local documents.
Commenters say the map covers a wide sky survey and links to a Legacy Survey Sky Viewer page that returned a 502 error.
Commenters say Codex now has a CLI and desktop app, with generous usage on many plans.
Commenters say Qwen 3.8 27B runs well locally on a MacBook Pro and can reach 150+ tokens per second on a 5090.
ForgeWM turns a bidirectional action-conditioned video generator into a few-step world model using causal training, distillation, and on-policy distribution matching.
The paper studies iterative test-time improvement loops where models accumulate self-feedback or environmental feedback across math, coding, and knowledge tasks.
SkillEvo uses multi-turn interaction feedback to keep evolving agent skills after single-turn evaluation stalls.
4DAnyone reconstructs 4D humans from uncalibrated monocular video by generating multiview-consistent views and lifting them into 4D Gaussian Splatting.
The paper tests ASR models with reference-disagreement, masked-number recovery, and orthographic-switching probes for benchmark optimization.
Across 37 tasks, the best LLM and best embedding model differ by 0.4 points, but the LLM route is more expensive.
GOAG learns a compact latent representation of a gripper's contact surface distribution to sample valid grasps without object-specific training.
CoToGrasp synthesizes stable grasps conditioned on contact topologies using an object-agnostic canonical workspace.
PolicyGuide compiles policy into a workflow graph and uses a proactive verifier at user-turn boundaries to guide agent behavior.
The method trains audio encoders to predict the next patch embedding, instead of using a static feature-extractor setup.
Jerry Murdock discusses credit-market warning signs, a possible dislocation in neo-clouds, and millions of specialized models versus a few general models.
The discussion covers replaying encrypted reasoning blobs across users and sibling models to recover hidden reasoning in plain text.
Safia Abdalla says Warp routes issues and pull requests through agents that triage, research, ask follow-up questions, and review code before humans are pinged.
Sebastian Fox reports roughly one in 20 ambient scribe notes contained an error serious enough to cause harm, and more than one in 10 had a hallucination.
This FP8-quantized checkpoint uses block size 128 and says performance metrics are nearly identical to the original model.
Unsloth says its Dynamic v3.0 GGUFs improve top-1% accuracy by 10% over other providers at the same size.
The GGUF release keeps the MTP head verified and says refusal behavior is substantially reduced, not eliminated.
DFlash 2 is a speculative-decoding drafter that predicts a block of tokens in one pass and preserves greedy output exactly.
The model is tagged for image-to-video, text-to-video, video-to-video, and audio-to-video generation.
MiniMax H3 generates video with native stereo audio at up to 2K resolution and 15-second duration.
The release includes 1,440 de novo miniprotein binders against 16 targets designed by two Claude models and characterized by two CROs.
The dataset contains 1,021.64 hours of recorded CAD, BIM, structural-analysis, and visualization workflows across 597 jobs.
The platform evaluates long-term memory systems with fixed Answer, Eval, datasets, models, configurations, review, and publication steps.
S1-mini is a 0.6B-parameter English text normalizer that rewrites raw ASR transcripts into clean written text and fits in a 462 MiB quantized build.
Ultra-FineWeb-L1 is a filtered English Common Crawl corpus built with main-text extraction, language filtering, MinHash deduplication, and custom cleaning.
OpenAI's Codex CLI runs locally on your computer and has installers for Mac, Linux, and Windows.
Claude Code runs in the terminal, understands a codebase, and can execute routine tasks, explain code, and handle git workflows.
The repository contains standalone Cursor plugin directories with .cursor-plugin/plugin.json manifests.
n8n combines a visual canvas with custom code and can be self-hosted or run in the cloud.
PostHog's self-driving mode turns product-data signals into researched reports and pull requests for review.
AI-Infra-Guard combines ClawScan, Agent Scan, AI infrastructure vulnerability scanning, MCP Server and Agent Skills scanning, and jailbreak evaluation.
The repo hosts open-source Modular Platform components, including the MAX framework and Mojo language.
Plane tracks issues, cycles, and product roadmaps and can be self-hosted or used in Plane Cloud.
OpenLogi is a local-first Rust alternative to Logitech Options+ that runs on macOS, Linux, and Windows.
The repository packages a single CLAUDE.md file derived from Andrej Karpathy's observations on LLM coding pitfalls.
The repository packages small, composable agent skills intended to work with any model and to support real engineering workflows.
ECC ships skills, agents, commands, and plugin-managed hooks, and the install instructions say to use verified channels only.
Superpowers is a software development methodology for coding agents built from composable skills and startup instructions.
Sub2API is an API gateway for distributing subscription quota across upstream providers such as Anthropic.
The same feed, delivered daily at 8 PM ET. No spam, unsubscribe anytime.