From 1a63c2b3a150187af3bfe84f51ed44f853771719 Mon Sep 17 00:00:00 2001 From: Lan_zhijiang Date: Sun, 20 Sep 2026 15:48:48 +0800 Subject: [PATCH 01/18] =?UTF-8?q?docs(self-hosted):=20=E5=A2=9E=E5=8A=A0?= =?UTF-8?q?=E9=9D=A2=E5=90=91=E7=94=A8=E6=88=B7=E7=9A=84=E8=87=AA=E9=83=A8?= =?UTF-8?q?=E7=BD=B2=E5=85=A5=E9=97=A8=E6=8C=87=E5=8D=97?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 提供 Neon + Render/Heroku 快速部署与手动部署路径 - 串联 CLI、信息源采集、索引检索和日常客户端使用 - 新增 Self-Hosted 导航与概览 --- website/.vitepress/config.mts | 10 + website/README.md | 4 +- website/content/en/index.md | 10 +- .../content/en/self-hosted/getting-started.md | 584 ++++++++++++++++++ website/content/en/self-hosted/index.md | 32 + 5 files changed, 635 insertions(+), 5 deletions(-) create mode 100644 website/content/en/self-hosted/getting-started.md create mode 100644 website/content/en/self-hosted/index.md diff --git a/website/.vitepress/config.mts b/website/.vitepress/config.mts index 24bd4d9..52e9d00 100644 --- a/website/.vitepress/config.mts +++ b/website/.vitepress/config.mts @@ -49,11 +49,21 @@ export default defineConfig({ }, themeConfig: { nav: [ + { text: 'Self-Hosted', link: '/self-hosted/' }, { text: 'Developer', link: '/developer/' }, { text: 'About', link: '/about/' }, { text: 'GitHub', link: 'https://github.com/InKCre' }, ], sidebar: { + '/self-hosted/': [ + { + text: 'Self-Hosted', + items: [ + { text: 'Overview', link: '/self-hosted/' }, + { text: 'Getting Started', link: '/self-hosted/getting-started' }, + ], + }, + ], '/developer/': [ { text: 'Developer Guide', diff --git a/website/README.md b/website/README.md index d3a8492..77b7f19 100644 --- a/website/README.md +++ b/website/README.md @@ -29,8 +29,8 @@ pnpm --dir website audit --audit-level high - Future Chinese source will live under `content/zh/` and be published under `/zh/`. - Only English is active until the Chinese route set is complete or the locale switch has a deliberate fallback. -- The current published routes are `/`, `/developer/`, `/developer/architecture`, - `/developer/contributing`, and `/about/`. +- The current routes are `/`, `/self-hosted/`, `/self-hosted/getting-started`, `/developer/`, + `/developer/architecture`, `/developer/contributing`, and `/about/`. - Section indexes use trailing-slash routes, such as `/developer/`. - Leaf pages use lowercase ASCII kebab-case routes without an extension, such as `/developer/architecture`. diff --git a/website/content/en/index.md b/website/content/en/index.md index f5e0fe1..c6a22fa 100644 --- a/website/content/en/index.md +++ b/website/content/en/index.md @@ -13,6 +13,9 @@ hero: indexing, and downstream use. actions: - theme: brand + text: Self-Hosted + link: /self-hosted/ + - theme: alt text: Developer Guide link: /developer/ - theme: alt @@ -36,9 +39,10 @@ features: InKCre is evolving rapidly. Product behavior, interfaces, and developer contracts may change as the project tests and solidifies its foundations. -This site currently focuses on a concise [Developer Guide](/developer/) and the project's -[identity and direction](/about/). It does not yet present a stable User Manual or a complete -third-party development platform. +Want to run your own instance? Start with [Self-Hosted](/self-hosted/) to choose a deployment path, +then follow its Getting Started guide to collect and use your information. The +[Developer Guide](/developer/) is for contributors; [About InKCre](/about/) explains the project's +identity and direction. ## Start with the current foundations diff --git a/website/content/en/self-hosted/getting-started.md b/website/content/en/self-hosted/getting-started.md new file mode 100644 index 0000000..8a8a95c --- /dev/null +++ b/website/content/en/self-hosted/getting-started.md @@ -0,0 +1,584 @@ +--- +title: Getting Started +description: + Create your own InKCre instance, collect your first feed, connect personal sources, and find your + information from the web or your everyday tools. +--- + +# Getting Started + +This is the [Self-Hosted](/self-hosted/) walkthrough: you will operate your own InKCre instance. It +is not a requirement that every InKCre user deploy a server. + +You have information scattered across subscriptions, saved repositories, messages, and email. This +guide takes you from **no InKCre instance** to collecting a source you care about, finding something +you collected, and making that collection available in your daily workflow. + +Start with one RSS feed. Once it works, add your personal sources one at a time. You do not need to +develop InKCre or configure an AI model for this first journey. The fork-based deployment paths also +avoid running Docker locally. You will use a browser, a text editor, and a few terminal commands. + +InKCre is under active development. Some setup still uses its command-line tool rather than a setup +wizard. Collection preserves source relationships; further AI organization needs its own +configuration. There is no default daily digest or automatic Telegram/email notification service. + +## 1. Create your instance + +Your instance stores your information and runs collection. The public +[Web app](https://app.inkcre.dev/settings) is a client you connect to it; opening the app does not +create a private instance for you. + +Choose one deployment path. **Neon + Render and Neon + Heroku are alternative fork-based quick +deployments**, not requirements of InKCre itself. Both workflows initialize the database and run +Core and PostgREST for you. If you prefer your own server or database, follow +[manual deployment](#manual-deployment) instead, then return to step 2. + +For either quick deployment, use GitHub, Neon, and one compute provider: + +| Account | What it does | +| ---------------- | --------------------------------------------------------------------------------------------------------------- | +| GitHub | Keeps your fork of InKCre and runs its deployment workflow. A fork is your own repository copy. | +| Neon | Stores your info-base in a PostgreSQL database. | +| Render or Heroku | Runs Core, which collects and processes information, and PostgREST, which connects the Web app to the database. | + +Use a dedicated Neon project for this instance and a password manager to retain credentials. An API +key lets the deployment workflow act on your hosting account; it is different from your sign-in +password. + +### Quick deployment: Render and Neon {#render-neon} + +1. Open [InKCre/core-py](https://github.com/InKCre/core-py), choose **Fork**, and create your + repository copy. Open its **Actions** tab and enable workflows if GitHub asks. +2. Create a project in [Neon](https://console.neon.tech). Keep the default `neondb` database and + `neondb_owner` role. Save the project ID and create a Neon API key with access to it. +3. Create a workspace in [Render](https://dashboard.render.com). Save its workspace ID from Settings + and create a Render API key. Your fork must be public, or Render must already have permission to + read it. +4. In your fork, open **Settings → Secrets and variables → Actions**. Use **New repository secret** + under **Secrets** and **New repository variable** under **Variables** to add: + +| Kind | Name | Value | +| -------- | ----------------------- | ------------------------------------------------------------------------------------ | +| Secret | `NEON_API_KEY` | Your Neon API key. | +| Secret | `RENDER_API_KEY` | Your Render API key. | +| Secret | `JWT_SECRET` | A new random secret of at least 32 ASCII characters, saved in your password manager. | +| Variable | `NEON_PROJECT_ID` | Your Neon project ID. | +| Variable | `RENDER_OWNER_ID` | Your Render workspace ID. | +| Variable | `RENDER_SERVICE_PREFIX` | A unique lowercase prefix such as `alex-inkcre`, between 3 and 40 characters. | + +5. Open **Actions → Deploy self-hosted InKCre → Run workflow**, select your fork's `main` branch, + and run it. Wait for completion. If it fails, open the failed step, correct the reported problem, + and rerun using the same credentials. +6. Open the completed run's summary. Save the **Core URL**, **PostgREST URL**, and **Core Peer ID**. + The workflow deliberately does not print your secret; retain the original value. +7. Open the Core URL with `/readyz` appended. Continue when it returns HTTP `200`. A sleeping + service may take time to start. + +**Checkpoint:** deployment succeeded, and you have both service URLs and your `JWT_SECRET`. Use the +**Core URL** for the CLI and the **PostgREST URL** for the Web app later. + +The workflow selects Render Free services. They sleep when idle, share account usage limits, and +cannot guarantee continuous collection. Check +[Render's current limits](https://render.com/docs/free) and your Neon plan before relying on them. +Continuous scheduled collection needs hosting that keeps Core running. The maintained deployment +procedure and recovery details live in the +[Core self-hosting guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/render-neon-self-host.md). + +### Quick deployment: Heroku and Neon {#heroku-neon} + +Choose this instead of the Render steps; you do not need both providers. The database still lives in +Neon, while Heroku runs two apps: Core and PostgREST. + +1. Fork `core-py` and enable Actions. Create a dedicated Neon project, keeping `neondb` and + `neondb_owner`, and retain the project ID and API key as described above. +2. Create a [Heroku account](https://dashboard.heroku.com/) with billing enabled and obtain an API + key. The workflow creates the two apps; you do not need to provision a Heroku database add-on. +3. In your fork's **Settings → Secrets and variables → Actions**, add the following settings. + Generate independent random values for the three credential secrets and retain them in your + password manager. + +| Kind | Name | Value | +| -------- | ----------------------------- | ---------------------------------------------------------------------------------- | +| Secret | `NEON_API_KEY` | Your Neon API key. | +| Secret | `HEROKU_API_KEY` | Your Heroku API key. | +| Secret | `JWT_SECRET` | A new random secret of at least 32 ASCII characters. | +| Secret | `CORE_DATABASE_PASSWORD` | A separate random password of at least 32 ASCII characters. | +| Secret | `POSTGREST_DATABASE_PASSWORD` | Another random password of at least 32 ASCII characters. | +| Variable | `NEON_PROJECT_ID` | Your Neon project ID. | +| Variable | `HEROKU_APP_PREFIX` | A unique lowercase app prefix, such as `alex-inkcre`, between 3 and 18 characters. | + +4. Open **Actions → Deploy self-hosted InKCre to Heroku → Run workflow**, select your fork's `main` + branch, and wait for the run to succeed. +5. Save the Core URL, PostgREST URL, and Core Peer ID from its summary. Open Core `/readyz` and wait + for HTTP `200`, then continue to step 2 below. + +The workflow runs one Eco web dyno per app. Heroku charges and +[Eco sleep behavior](https://devcenter.heroku.com/articles/eco-dyno-hours) apply. Keep the same +database passwords on reruns; replacing them is a coordinated credential rotation, not a routine +redeploy. As with Render Free, sleeping Core cannot provide continuous collection. The maintained +deployment procedure and recovery details live in the +[Core Heroku self-hosting guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/heroku-neon-self-host.md). + +### Manual deployment {#manual-deployment} + +You can use your own machine, VPS, containers, or other hosting. No GitHub fork, Neon account, +Render account, or Heroku account is inherently required. These providers automate a portable +runtime consisting of: + +| Component | Responsibility | +| ----------------------------------------------------------- | --------------------------------------------------------------------------------------- | +| PostgreSQL with the required extensions, including pgvector | Persists the info-base and shared configuration. | +| Database initializer | Creates/migrates the schema, provisions runtime roles, and reconciles built-in records. | +| PostgREST | Exposes the admitted database API used by the Web client. | +| Core Python program | Runs collection, Jobs, Extensions, and the Core HTTP API. | + +The initializer is a deployment step, not an additional always-running service. Core and PostgREST +connect to the same initialized database with different runtime roles. The CLI connects to Core; the +Web client uses PostgREST and discovers Core capabilities. The Web app itself can remain at +`app.inkcre.dev` or be hosted separately. + +For a manual installation, work through these steps using the runtime documentation for the Core +revision you selected: + +1. Provision PostgreSQL with the required extensions. Retain an owner connection for initialization + and migrations; do not use that privileged connection as the application's runtime connection. +2. Run Core's ordered database initializer with the runtime profile. It provisions schema and roles + as well as migrations: starting an empty database and only launching Python is not enough. The + [database lifecycle guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/database-contract.md) + owns the initializer commands, credentials, and readiness checks. +3. Configure PostgREST with the `authenticator` connection, the admitted `inkcre` schema, and the + deployment's JWT settings. Configure Core with the `inkcre_core` connection and matching JWT + settings. Keep the migration-owner credentials out of both running services. +4. Start the Core Python program with its pinned dependencies, or use its container image, and start + PostgREST. The [Core README](https://github.com/InKCre/core-py#readme) and + [container/runtime guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/docker.md) + own the supported entry points. The Compose stack is a useful topology reference, but its + development credentials and defaults are not a production configuration. +5. Configure HTTPS and reachable service URLs, advertise Core's public address in its Peer + configuration, and arrange process restarts, backups, and upgrades. Verify Core `/readyz` and + authenticated database access before connecting clients. The + [runtime orchestration guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/runtime-orchestration.md) + explains readiness and capability availability. + +**Checkpoint for every deployment path:** you have a ready Core URL, a PostgREST URL, your private +JWT secret, and a persistent Core Peer identity. Continue below; collecting and using information +works the same way regardless of the hosting provider. + +**Keep your instance credentials private.** `JWT_SECRET` grants control of the instance, not a +limited personal login. Do not paste it into an online JWT generator, an issue, a chat, or your +public fork. This deployment does not isolate different users from one another. Start with your own +information and trusted devices. + +## 2. Connect the command-line tool {#connect-cli} + +The CLI handles setup operations that do not yet have a Web form. It connects over HTTPS; it does +not run another server on your computer. + +1. Install [Python](https://www.python.org/downloads/) 3.12 or later if needed. Check + `python --version` in a terminal; use `python3` if that is your system's command name. +2. Create a local environment: + + ```sh + python -m venv .venv + ``` + + Activate it with `source .venv/bin/activate` on macOS/Linux, or `.venv\Scripts\Activate.ps1` in + Windows PowerShell. Then install the published CLI: + + ```sh + python -m pip install inkcre-cli + inkcre-cli --help + ``` + +3. In a local folder outside any Git repository, use a text editor to create `connection.json`: + + ```json + { + "base_url": "https://YOUR-CORE-HOST", + "jwt_secret": "YOUR-SAVED-JWT-SECRET" + } + ``` + + Replace both values, preserving the quotes. Use the Core base URL without `/readyz`. This file + contains a credential; keep it private. + +4. From that folder, save and check the connection: + + ```sh + inkcre-cli connection set personal --input connection.json + inkcre-cli connection use personal + inkcre-cli connection check + ``` + + Both readiness and the authenticated read should succeed. You can remove the temporary + `connection.json` afterward: the CLI retains it at `.inkcre/cli/connections.json` in your home + directory. Protect that file too. + +**Checkpoint:** `connection check` can read your instance. For `401`, check the secret; for a +connection failure, check the Core URL and wake `/readyz`. In later terminal sessions, reactivate +the environment before using `inkcre-cli`. + +The [CLI reference](https://github.com/InKCre/core-py/blob/main/cli/README.md) owns command details. +`--help` explains a command; `--schema` on input-taking commands shows the configuration accepted by +your running instance. + +## 3. Collect your first RSS feed + +Choose a publication you already read and copy its **RSS or Atom feed URL**. Its normal homepage URL +is not usually a feed URL. Look for an RSS/Subscribe link on the publication. + +### Enable the collector + +Install the RSS Extension on Core, then enable it: + +```sh +inkcre-cli extension install inkcre/rss --version 0.2.0 +inkcre-cli extension enable inkcre/rss +inkcre-cli source types +``` + +Version `0.2.0` is a published Python release for Core Host `0.2.x`. For a different Host version, +check the [RSS release listing](https://registry.inkcre.dev/v1/extensions/inkcre/rss) for a +published release with a compatible `python.host_sdk_version`. Do not guess a version or use +`latest`. If already installed, inspect `inkcre-cli extension get inkcre/rss` before changing it. + +The type list should contain `extensions.rss.rss.Source` and `extensions.rss.atom.Source`. The Web +app's Extensions switch controls its browser runtime; it does not enable this Core collector. + +### Add and run the source + +1. Create `feed.json` in your local working folder: + + ```json + { + "nickname": "My first feed", + "config": { + "feed_url": "https://YOUR-PUBLICATION/FEED", + "fetch_full_text": false, + "download_enclosures": false + } + } + ``` + + Replace the URL. These first-run settings collect feed content without extra article fetches or + attachment downloads. + +2. Create the Source, choosing the matching feed format: + + ```sh + inkcre-cli source create --type extensions.rss.rss.Source --input feed.json + ``` + + For Atom, use `extensions.rss.atom.Source`. Record the returned Source `id`. + +3. Collect it. Replace `42` with your Source ID: + + ```sh + inkcre-cli source collect 42 --input-json '{}' + ``` + +4. The response contains a **Job ID**, different from the Source ID. Replace `17` with it: + + ```sh + inkcre-cli job wait 17 --for 30s + ``` + + Look for `status: finished`. If still `pending` or `running`, observe again rather than creating + another collect job. If `failed`, inspect `inkcre-cli job get 17 --json` and its diagnostics. + Ending observation does not stop the Job. + +**Checkpoint:** the collection Job finished. A feed exposes only what its publisher currently +provides; success does not mean its entire historical archive was imported. + +## 4. Find and inspect what you saved + +Collection and indexing are separate. Create a lexical-index maintenance Job after collection to +enable search by remembered words, without an AI provider: + +```sh +inkcre-cli job create --type core.feature_retrieval.lexical.maintain.v1 --input-json '{"parameters":{}}' +``` + +Use its returned Job ID with `inkcre-cli job wait JOB_ID --for 30s`, replacing `JOB_ID` with the +number. Check its status and `state` for indexed records and diagnostics. Maintenance processes a +bounded batch; repeat if you imported more than one batch can index. + +Copy a distinctive phrase from an item in your feed and search: + +```sh +inkcre-cli recall 'A phrase from your feed item' --mode lexical +``` + +Read a returned Block's content and relationships, replacing `123` with that Block's ID: + +```sh +inkcre-cli resolver invoke block:123 --method get_text +inkcre-cli graph neighborhood block:123 +``` + +**Checkpoint:** you can retrieve a real item you recognize and see its source relationships. You +have completed the first loop: source → saved information → information you can use. + +This first form of organization comes from the source: feed items belong to feeds, GitHub +repositories can belong to Lists, and mail has sender and mailbox relationships. It does not mean +InKCre has already summarized, tagged, or reorganized everything with AI. + +## 5. Keep collecting and searching + +After the manual run works, create `collect-hourly.json`, replacing `42` with your Source ID: + +```json +{ + "schedule": "0 * * * *", + "job_parameters": { "source": 42, "config": {} } +} +``` + +```sh +inkcre-cli cron create --job-type core.source.collect.v1 --input collect-hourly.json +``` + +This five-field schedule means “at minute zero of every hour.” Add independent index maintenance so +later items become searchable. Save this as `index-periodically.json`: + +```json +{ + "schedule": "*/10 * * * *", + "job_parameters": {} +} +``` + +```sh +inkcre-cli cron create --job-type core.feature_retrieval.lexical.maintain.v1 --input index-periodically.json +inkcre-cli cron list +``` + +Create each schedule once. Inspect its returned Cron ID with `inkcre-cli cron get ID`; `last_job` +identifies the Job to check. Pause it with `inkcre-cli cron disable ID`. Replace `ID` with the +actual number. Your terminal may be closed, but Core must run. Sleeping hosts miss occurrences and +do not automatically catch up. Independent indexing also means new items may not become searchable +immediately. + +## 6. Add the sources you actually use + +For each new source: enable its Core Extension, create the Source, collect once, inspect the Job, +run index maintenance, and search for a known item. Only then add a schedule. + +The versions below are published releases for Core Host `0.2.x`. Check the linked release listings +and your running type's `--schema` when versions differ. Source files and command output may contain +account credentials; do not publish them. + +### GitHub Stars and Lists + +1. Create a personal access token for the account whose Stars and Lists you want to collect, + following + [GitHub's token instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens). + Its account and permissions determine which data is visible. +2. Enable the Extension: + + ```sh + inkcre-cli extension install inkcre/github --version 0.3.0 + inkcre-cli extension enable inkcre/github + ``` + +3. Save `github.json` with your token: + + ```json + { + "nickname": "My GitHub saves", + "config": { "github_token": "YOUR-GITHUB-TOKEN" } + } + ``` + + ```sh + inkcre-cli source create --type extensions.github.stars.Source --input github.json + ``` + +4. Collect the returned Source ID as in step 3. After indexing, search for a repository you starred. + Each run synchronizes current Stars and Lists; it is not a code backup or a full GitHub activity + archive. + +See the [GitHub collector](https://github.com/InKCre/core-py/blob/main/extensions/github/README.md) +and [published releases](https://registry.inkcre.dev/v1/extensions/inkcre/github). + +### Email over IMAP + +1. Find your provider's IMAP hostname and enable IMAP if required. Obtain an app-specific password + where supported. Accounts requiring an unsupported authentication method cannot be connected by + substituting an ordinary login password. +2. Install and enable `inkcre/mail` version `0.3.0`, using the two Extension commands above with + `inkcre/mail` in place of `inkcre/github`. +3. Save `mail.json`, replacing the host and credentials: + + ```json + { + "nickname": "My mail", + "config": { + "protocol": "imap", + "parameters": { + "host": "YOUR-IMAP-HOST", + "port": 993, + "security": "tls", + "username": "YOUR-MAIL-LOGIN", + "password": "YOUR-APP-PASSWORD" + }, + "ordinary_mark_as_seen": false, + "synchronize_deletions": false + } + } + ``` + + ```sh + inkcre-cli source create --type extensions.mail.source.Source --input mail.json + ``` + +4. Send yourself a test email **after creating the Source**, then collect its ID. Ordinary + collection starts with new mail; an empty first run does not imply login failed. Inspect mailbox + diagnostics even if the Job finishes. This example preserves unread state; the collector's + default would mark ordinary collected mail as seen. +5. For older mail, request a small explicit backfill. Replace `42` with the Mail Source ID and + choose dates for your mailbox: + + ```sh + inkcre-cli source backfill 42 --input-json '{"since":"2026-09-01","before":"2026-09-08"}' + ``` + + The start date is included and the end date excluded. Observe the Job and index afterward. This + collector does not send mail or create email digests. + +See the +[Mail configuration](https://github.com/InKCre/core-py/blob/main/docs/30-unit-tdd/mail-extension.md) +and [published releases](https://registry.inkcre.dev/v1/extensions/inkcre/mail). + +### Send or forward messages from Telegram + +1. Create a dedicated bot with [@BotFather](https://t.me/botfather): send `/newbot`, follow its + naming prompts, and retain the token. Obtain your own numeric user ID using + [Telegram Desktop's data export](https://telegram.org/blog/export-and-more): open **Settings → + Advanced → Export Telegram data**, include personal information, and choose JSON format. In the + exported JSON, use `personal_information.user_id`, not your `@username` or the bot's ID. Telegram + may require confirmation from another device or a waiting period. +2. Install and enable `inkcre/telegram` version `0.3.0` with the same Extension commands. +3. Save `telegram.json`, replacing the token and example user ID: + + ```json + { + "nickname": "My Telegram inbox", + "config": { + "bot_token": "YOUR-BOT-TOKEN", + "bound_user_id": 123456789, + "download_attachments": false + } + } + ``` + + ```sh + inkcre-cli source create --type extensions.telegram.source.Source --input telegram.json + ``` + +4. Send a distinctive text message in a private chat with the bot, then collect the Source ID. + Successfully saved messages receive a 👍 reaction. After indexing, search for its text. +5. Add a schedule to forward messages without running the CLI each time. `*/5 * * * *` polls every + five minutes while Core is awake. Telegram retains updates for a limited time, so a long-sleeping + instance should not be your only copy. + +Use one bot for one Source. This is a capture inbox, not group/channel history import or a +notification destination. Attachments are metadata-only in this example; enable +`download_attachments` when you want their bytes saved too. See the +[Telegram collector](https://github.com/InKCre/core-py/blob/main/extensions/telegram/README.md) and +[published releases](https://registry.inkcre.dev/v1/extensions/inkcre/telegram). + +Add more RSS/Atom subscriptions with one Source per feed. Memos-compatible capture is also available +through a separately configured +[Memos Extension](https://github.com/InKCre/core-py/blob/main/docs/30-unit-tdd/memos-extension.md). +An arbitrary service or browser account is not automatically a supported Source; check for a +collector before assuming its history can be imported. + +## 7. Bring information into your daily workflow + +### Read and explore in the Web app + +The browser needs its **own** Client ID. Do not reuse the Core Peer ID: the browser excludes its own +identity when looking for another Peer to execute a capability. Settings does not yet create this +record for you. + +1. In your dedicated Neon project's **SQL Editor**, select the same default branch and `neondb` + database as deployment. For a manually hosted database, use your PostgreSQL SQL client against + the initialized InKCre database instead. Run this once to register your browser: + + ```sql + INSERT INTO inkcre.peers (id, name, config) + VALUES ( + gen_random_uuid(), + 'My web browser', + '{"extension_registry_url":"https://registry.inkcre.dev"}'::jsonb + ) + RETURNING id; + ``` + + Save the returned UUID as your browser Client ID. This adds one record without replacing Core's + record. Reuse it when reconnecting this browser. + +2. Wake Core `/readyz`, then open [Web app Settings](https://app.inkcre.dev/settings). +3. Enter **PostgreSQL REST URL** = PostgREST base URL, **JWT Secret** = your saved secret, and + **Client ID** = the new browser UUID. Choose **Save**, then reload. The Clients list should + include Core, reporting online. +4. Open **Info Base**, search for the phrase that worked in step 4, and inspect a result. Use **View + content** for supported content and the graph view to follow relationships. The list is a search + surface, not every stored Block. Some source renderers need a compatible Web Extension; the CLI's + `get_text` remains a way to read Core-resolved text. +5. Bookmark the app. Another browser/device needs its own connection setup. **Export** omits the + secret and is not an info-base backup. + +### Use your information from a terminal or AI tool + +Keep `inkcre-cli recall 'your clue' --mode lexical` available wherever you work. A terminal-based +assistant you trust can use the installed CLI and named connection. For example: + +> Use my InKCre personal connection to find articles I saved about distributed systems. Read the +> matches and cite their original links. Do not change my sources or data. + +The connection has owner authority; only give it to a tool you trust. Its model provider may receive +the content it reads. + +For clients that connect over MCP instead of running terminal commands, follow the +[MCP Sink setup](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/mcp-sink.md). Create +and enable a `core.mcp.v1` Sink with a separate PAT, then configure Bearer authentication for +`https://YOUR-CORE-HOST/sinks/SINK_ID/mcp`. This needs explicit Sink and client setup; the CLI has +no `sink` command. The MCP PAT is not `JWT_SECRET`. Check the client's authentication support or the +documented tunnel before choosing this route. + +These paths make your saved information available on demand. They do not configure proactive +notifications or a daily briefing. + +### Add AI organization when you need it + +Once collection and reading work, configure a model provider and Agent for rumination, then +explicitly reconsider a Block. Follow the +[Core organization configuration](https://github.com/InKCre/core-py/blob/main/docs/30-unit-tdd/organization.md) +for the Agent and deployment settings before using **Ruminate** or +`inkcre-cli organization ruminate BLOCK_ID`. + +Inspect the Job and graph afterward. A valid result may add nothing. Re-index after new content is +added to find it by words. Semantic search additionally needs an embedding provider/profile and +maintenance; an LLM API key alone does not enable it. Selected content leaves your instance for +configured AI providers and may incur charges. + +## If a step does not work + +| What you observe | What to check next | +| ---------------------------------------- | --------------------------------------------------------------------------------------------------------- | +| Deployment workflow missing | Enable Actions in your fork and check that it is up to date with upstream `main`. | +| Core slow or offline | Open Core `/readyz`; inspect host logs if it stays unready. PostgREST may sleep separately. | +| Client returns `401` | Check endpoint and secret. Anonymous PostgREST `401` is expected; authenticated `401` is not. | +| Source type absent | Install a compatible Extension and enable it on Core, not only in the Web app. | +| Job stays pending | Confirm Core is awake and its collector enabled. Job creation only confirms acceptance. | +| Collection finished but search empty | Confirm the source exposed the item; run lexical maintenance and inspect diagnostics. | +| CLI works but Web cannot find a provider | Check Core readiness and that browser and Core Peer IDs differ. | +| Old mail missing | Ordinary collection starts with new mail. Request a historical backfill. | +| Truncated content or no attachment bytes | First-run examples avoid extra downloads. Check the collector's settings. | +| Request outcome uncertain | Read the Job or resulting data before repeating a write; a lost response does not prove nothing happened. | + +Keep hosting and connection credentials in your password manager. Review your database's +backup/restore options and hosting usage before collecting irreplaceable information. A fork, a +client configuration export, and a source account are not backups of your info-base. diff --git a/website/content/en/self-hosted/index.md b/website/content/en/self-hosted/index.md new file mode 100644 index 0000000..8325c2c --- /dev/null +++ b/website/content/en/self-hosted/index.md @@ -0,0 +1,32 @@ +--- +title: Self-Hosted +description: Choose how to run your own InKCre instance and connect it to your information sources. +--- + +# Self-Hosted + +Self-hosting means operating your own InKCre instance: you choose where information is stored, which +sources it collects, and which clients can access it. You also manage credentials, hosting +availability, backups, and updates. This section is for users choosing that responsibility. + +## Choose a deployment path + +| Path | What you manage | Start here | +| ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | +| Fork quick deployment: Neon + Render | Provider accounts and GitHub settings; the workflow initializes the database and deploys Core and PostgREST. | [Render walkthrough](/self-hosted/getting-started#render-neon) | +| Fork quick deployment: Neon + Heroku | The same topology on Heroku, with its billing and runtime settings. | [Heroku walkthrough](/self-hosted/getting-started#heroku-neon) | +| Manual deployment | PostgreSQL initialization, PostgREST, the Core Python process or container, and hosting operations on infrastructure you choose. | [Manual deployment overview](/self-hosted/getting-started#manual-deployment) | + +Neon, Render, and Heroku are convenient deployment options, not product dependencies. Forking +provides a ready-made deployment workflow; it is not required to run InKCre. Both quick deployments +produce your own instance rather than access to a shared hosted account. Their sleeping compute +plans need particular care if you want continuous collection. + +## From an empty instance to useful information + +Follow [Getting Started](/self-hosted/getting-started) to deploy an instance, collect one RSS feed, +find a saved item, and add the sources you use. The guide then connects the instance to the Web app +and everyday tools. It assumes basic technical familiarity but no InKCre development setup. + +If you already operate a compatible instance, begin at +[connecting the CLI](/self-hosted/getting-started#connect-cli). From a1928a395be99c1979f6358cc0c5d302779c7b35 Mon Sep 17 00:00:00 2001 From: Lan_zhijiang Date: Sun, 20 Sep 2026 15:56:04 +0800 Subject: [PATCH 02/18] =?UTF-8?q?fix(preview):=20=E5=B0=86=E9=A2=84?= =?UTF-8?q?=E8=A7=88=E7=8A=B6=E6=80=81=E5=92=8C=E5=85=A5=E5=8F=A3=E5=85=B3?= =?UTF-8?q?=E8=81=94=E5=88=B0=20PR=20=E6=8F=90=E4=BA=A4?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 在精确 PR head 上报告 pending、成功或错误状态 - 保留现有部署和清理流程，并说明 main 控制器生效边界 --- .github/workflows/pages-preview.yml | 37 +++++++++++++++++++++++++++++ website/README.md | 6 ++++- 2 files changed, 42 insertions(+), 1 deletion(-) diff --git a/.github/workflows/pages-preview.yml b/.github/workflows/pages-preview.yml index d5342b2..f2b415a 100644 --- a/.github/workflows/pages-preview.yml +++ b/.github/workflows/pages-preview.yml @@ -78,12 +78,28 @@ jobs: permissions: contents: read deployments: write + statuses: write runs-on: ubuntu-latest timeout-minutes: 15 environment: name: preview url: ${{ steps.pages.outputs.pages-deployment-alias-url }} steps: + - name: Mark the pull-request preview pending + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 + env: + PREVIEW_HEAD_SHA: ${{ needs.identity.outputs.head_sha }} + with: + script: | + await github.rest.repos.createCommitStatus({ + ...context.repo, + sha: process.env.PREVIEW_HEAD_SHA, + context: 'docs preview', + state: 'pending', + description: '正在构建并部署预览', + target_url: `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`, + }) + - name: Checkout the trusted preview controller uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: @@ -142,3 +158,24 @@ jobs: CLOUDFLARE_PAGES_DEPLOYMENT_ID: ${{ steps.pages.outputs.pages-deployment-id }} CLOUDFLARE_PAGES_DEPLOYMENT_URL: ${{ steps.pages.outputs.pages-deployment-alias-url }} INKCRE_PAGES_SMOKE_MODE: preview + + - name: Report the preview result on the pull-request commit + if: ${{ always() }} + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 + env: + PREVIEW_HEAD_SHA: ${{ needs.identity.outputs.head_sha }} + PREVIEW_RESULT: ${{ job.status }} + PREVIEW_URL: ${{ steps.pages.outputs.pages-deployment-alias-url }} + with: + script: | + const succeeded = process.env.PREVIEW_RESULT === 'success' + await github.rest.repos.createCommitStatus({ + ...context.repo, + sha: process.env.PREVIEW_HEAD_SHA, + context: 'docs preview', + state: succeeded ? 'success' : 'error', + description: succeeded ? '预览已就绪，点击查看' : '预览失败或取消，点击查看日志', + target_url: succeeded + ? process.env.PREVIEW_URL + : `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`, + }) diff --git a/website/README.md b/website/README.md index 77b7f19..e29864b 100644 --- a/website/README.md +++ b/website/README.md @@ -70,7 +70,11 @@ same-repository run, the trusted Preview workflow checks out that exact head, bu publishes an isolated, deterministic, short-lived preview. Fork pull requests receive no preview credentials, preview origins remain `noindex`, and closing the pull request replaces the live preview with a trusted closed-preview tombstone. The stable `preview-docs-pr-N` branch alias is the -user-facing preview URL and is recorded against the pull-request head in GitHub; Cloudflare retains +user-facing preview URL. The `docs preview` commit status on the exact pull-request head links to +that URL after deployment and smoke checks succeed; pending, failed, or cancelled runs link to the +workflow logs. The workflow's automatic environment deployment record belongs to its trusted +`main` controller, so the explicit commit status provides the PR-facing entry point. Changes to +this `workflow_run` controller take effect after merging into `main`. Cloudflare retains the underlying immutable deployments in its history. If automatic retirement fails, the cleanup workflow can be run manually for the closed pull-request number. A preview build is never promoted to production. From 706d49d3f4aaab37a83f872d3779b7ec703d21a4 Mon Sep 17 00:00:00 2001 From: Lan_zhijiang Date: Sun, 20 Sep 2026 15:56:15 +0800 Subject: [PATCH 03/18] =?UTF-8?q?docs(preview):=20=E6=A0=BC=E5=BC=8F?= =?UTF-8?q?=E5=8C=96=E9=A2=84=E8=A7=88=E5=8F=91=E5=B8=83=E8=AF=B4=E6=98=8E?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- website/README.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/website/README.md b/website/README.md index e29864b..c66b2e9 100644 --- a/website/README.md +++ b/website/README.md @@ -72,12 +72,11 @@ credentials, preview origins remain `noindex`, and closing the pull request repl preview with a trusted closed-preview tombstone. The stable `preview-docs-pr-N` branch alias is the user-facing preview URL. The `docs preview` commit status on the exact pull-request head links to that URL after deployment and smoke checks succeed; pending, failed, or cancelled runs link to the -workflow logs. The workflow's automatic environment deployment record belongs to its trusted -`main` controller, so the explicit commit status provides the PR-facing entry point. Changes to -this `workflow_run` controller take effect after merging into `main`. Cloudflare retains -the underlying immutable deployments in its history. If automatic retirement fails, the cleanup -workflow can be run manually for the closed pull-request number. A preview build is never promoted -to production. +workflow logs. The workflow's automatic environment deployment record belongs to its trusted `main` +controller, so the explicit commit status provides the PR-facing entry point. Changes to this +`workflow_run` controller take effect after merging into `main`. Cloudflare retains the underlying +immutable deployments in its history. If automatic retirement fails, the cleanup workflow can be run +manually for the closed pull-request number. A preview build is never promoted to production. Protected `main` is the publication authority. `Pages deployment` runs for a push to `main`; failed runs can be rerun for the same commit, while rollback starts by reverting `main` through a pull From 58800a86f5680c5c2d78c7f93483fb1a0957ff30 Mon Sep 17 00:00:00 2001 From: Lan_zhijiang Date: Sun, 20 Sep 2026 16:00:38 +0800 Subject: [PATCH 04/18] =?UTF-8?q?docs(onboarding):=20=E5=8C=BA=E5=88=86?= =?UTF-8?q?=E5=BA=94=E7=94=A8=E5=85=A5=E9=97=A8=E4=B8=8E=E8=87=AA=E9=83=A8?= =?UTF-8?q?=E7=BD=B2=E6=8C=87=E5=8D=97?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 首页 Getting Started 介绍 What、Why、How 与实例访问路径 - Self-Hosted 下区分 Getting Started 和 Advanced - 将手动部署迁至进阶指南并保留原入口跳转 --- website/.vitepress/config.mts | 6 +- website/README.md | 5 +- website/content/en/getting-started.md | 84 +++++++++++++++++++ website/content/en/index.md | 8 +- website/content/en/self-hosted/advanced.md | 79 +++++++++++++++++ .../content/en/self-hosted/getting-started.md | 42 +--------- website/content/en/self-hosted/index.md | 15 ++-- 7 files changed, 188 insertions(+), 51 deletions(-) create mode 100644 website/content/en/getting-started.md create mode 100644 website/content/en/self-hosted/advanced.md diff --git a/website/.vitepress/config.mts b/website/.vitepress/config.mts index 52e9d00..9af7b23 100644 --- a/website/.vitepress/config.mts +++ b/website/.vitepress/config.mts @@ -49,18 +49,20 @@ export default defineConfig({ }, themeConfig: { nav: [ - { text: 'Self-Hosted', link: '/self-hosted/' }, + { text: 'Getting Started', link: '/getting-started' }, { text: 'Developer', link: '/developer/' }, { text: 'About', link: '/about/' }, { text: 'GitHub', link: 'https://github.com/InKCre' }, ], sidebar: { - '/self-hosted/': [ + '/': [ + { text: 'Getting Started', link: '/getting-started' }, { text: 'Self-Hosted', items: [ { text: 'Overview', link: '/self-hosted/' }, { text: 'Getting Started', link: '/self-hosted/getting-started' }, + { text: 'Advanced', link: '/self-hosted/advanced' }, ], }, ], diff --git a/website/README.md b/website/README.md index c66b2e9..96ba509 100644 --- a/website/README.md +++ b/website/README.md @@ -29,8 +29,9 @@ pnpm --dir website audit --audit-level high - Future Chinese source will live under `content/zh/` and be published under `/zh/`. - Only English is active until the Chinese route set is complete or the locale switch has a deliberate fallback. -- The current routes are `/`, `/self-hosted/`, `/self-hosted/getting-started`, `/developer/`, - `/developer/architecture`, `/developer/contributing`, and `/about/`. +- The current routes are `/`, `/getting-started`, `/self-hosted/`, `/self-hosted/getting-started`, + `/self-hosted/advanced`, `/developer/`, `/developer/architecture`, `/developer/contributing`, and + `/about/`. - Section indexes use trailing-slash routes, such as `/developer/`. - Leaf pages use lowercase ASCII kebab-case routes without an extension, such as `/developer/architecture`. diff --git a/website/content/en/getting-started.md b/website/content/en/getting-started.md new file mode 100644 index 0000000..7b37de0 --- /dev/null +++ b/website/content/en/getting-started.md @@ -0,0 +1,84 @@ +--- +title: Getting Started +description: + Understand what InKCre does, why you might use it, and how to start with your information. +--- + +# Getting Started + +InKCre helps you turn information scattered across your tools into a collection you can return to +and use. Start here to understand the experience, then choose how you will access an instance. You +do not need to be an InKCre developer to begin. + +## What is InKCre? + +InKCre collects information, organizes it in an **info-base**, and makes it available for retrieval +and use in other tools. Think of the info-base as your reusable collection, including connections +between pieces of information rather than only a folder of copies. + +For example, you might collect articles from RSS feeds, keep track of saved GitHub repositories, and +capture messages you forward to a Telegram bot. Later, you can find an article from a phrase you +remember, follow its related information, or make it available to a trusted assistant. + +These are connected capabilities, not mandatory stages: you can collect and retrieve information +without first configuring AI organization. Extensions provide integrations with different sources +and tools; each integration has its own setup and limits. + +## Why use it? + +Saving information is useful only if you can find and reuse it. An article in one app, a repository +in another, and a note in a third can become difficult to bring together when you need them. + +InKCre gives that information a common home without making its usefulness depend on the original +collector or a single client. You choose the sources that matter, then access the collection from +the Web app, the command line, or connected tools in your workflow. Self-hosting also lets you +choose where the instance runs and where its information is stored. + +Start with a concrete need—such as finding useful articles from your subscriptions—rather than +connecting every source at once. One working source and a successful search are a better first +milestone than a large collection you cannot yet use. + +## How do I start? + +### 1. Choose how to access an instance + +An **instance** stores your info-base and runs capabilities such as collection. A **client**, such +as the [Web app](https://app.inkcre.dev/settings), connects to that instance. Opening the Web app +does not create an instance for you. + +- **Run your own instance:** choose [Self-Hosted](/self-hosted/). Its + [Getting Started](/self-hosted/getting-started) guide walks through quick deployment and your + first collection. [Advanced](/self-hosted/advanced) covers manual deployment and operating it on + infrastructure you choose. +- **Already have access to an instance:** obtain connection details from the person operating it, + then follow the [CLI connection steps](/self-hosted/getting-started#connect-cli) and subsequent + collection and client setup sections. Skip the deployment steps. Only connect information and + tools you are authorized to use with that instance. + +This guide does not assume a hosted sign-up service or separate private user accounts inside an +instance. Access arrangements and trust matter; do not treat a shared instance as an isolated +personal account. + +### 2. Collect one useful source + +Begin with a public RSS feed you care about. The self-hosted walkthrough takes you through +installing its Extension, adding the Source configuration, and checking that the collection Job +completed. Once that works, add personal sources one at a time, with the credentials and permissions +each requires. + +### 3. Find and use what you collected + +Maintain the search index, search for something you remember, and open a result. Then connect the +Web app or a tool you already use so that the collection is accessible in your daily work. The +walkthrough covers these steps after deployment; they apply to an existing compatible instance too. + +AI organization and semantic retrieval are optional next steps with their own provider and +maintenance setup. InKCre does not automatically configure a daily digest or Telegram/email push +notifications: making information available in your tools is distinct from proactively sending it. + +## Your next step + +If you do not have an instance yet, open [Self-Hosted](/self-hosted/) and choose a deployment path. +If you want to understand or contribute to the implementation, use the +[Developer Guide](/developer/) instead. For the project's values and direction, read +[About InKCre](/about/). diff --git a/website/content/en/index.md b/website/content/en/index.md index c6a22fa..3613760 100644 --- a/website/content/en/index.md +++ b/website/content/en/index.md @@ -13,8 +13,8 @@ hero: indexing, and downstream use. actions: - theme: brand - text: Self-Hosted - link: /self-hosted/ + text: Getting Started + link: /getting-started - theme: alt text: Developer Guide link: /developer/ @@ -39,8 +39,8 @@ features: InKCre is evolving rapidly. Product behavior, interfaces, and developer contracts may change as the project tests and solidifies its foundations. -Want to run your own instance? Start with [Self-Hosted](/self-hosted/) to choose a deployment path, -then follow its Getting Started guide to collect and use your information. The +Start with [Getting Started](/getting-started) to learn what InKCre does, why you might use it, and +how to begin. Running your own instance is one path, covered by [Self-Hosted](/self-hosted/). The [Developer Guide](/developer/) is for contributors; [About InKCre](/about/) explains the project's identity and direction. diff --git a/website/content/en/self-hosted/advanced.md b/website/content/en/self-hosted/advanced.md new file mode 100644 index 0000000..7f2a59b --- /dev/null +++ b/website/content/en/self-hosted/advanced.md @@ -0,0 +1,79 @@ +--- +title: Advanced Self-Hosting +description: + Deploy InKCre on your own infrastructure and plan its operation, upgrades, and recovery. +--- + +# Advanced Self-Hosting + +This guide is for people who want to manage the runtime directly or move beyond the fork-based quick +deployments. If you want the shortest route to your first collected information, start with +[Self-Hosted Getting Started](/self-hosted/getting-started). + +## Manual deployment + +You can use your own machine, VPS, containers, or other hosting. No GitHub fork, Neon account, +Render account, or Heroku account is inherently required. These providers automate a portable +runtime consisting of: + +| Component | Responsibility | +| ----------------------------------------------------------- | --------------------------------------------------------------------------------------- | +| PostgreSQL with the required extensions, including pgvector | Persists the info-base and shared configuration. | +| Database initializer | Creates/migrates the schema, provisions runtime roles, and reconciles built-in records. | +| PostgREST | Exposes the admitted database API used by the Web client. | +| Core Python program | Runs collection, Jobs, Extensions, and the Core HTTP API. | + +The initializer is a deployment step, not an additional always-running service. Core and PostgREST +connect to the same initialized database with different runtime roles. The CLI connects to Core; the +Web client uses PostgREST and discovers Core capabilities. The Web app itself can remain at +`app.inkcre.dev` or be hosted separately. + +For a manual installation, work through these steps using the runtime documentation for the Core +revision you selected: + +1. Provision PostgreSQL with the required extensions. Retain an owner connection for initialization + and migrations; do not use that privileged connection as the application's runtime connection. +2. Run Core's ordered database initializer with the runtime profile. It provisions schema and roles + as well as migrations: starting an empty database and only launching Python is not enough. The + [database lifecycle guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/database-contract.md) + owns the initializer commands, credentials, and readiness checks. +3. Configure PostgREST with the `authenticator` connection, the admitted `inkcre` schema, and the + deployment's JWT settings. Configure Core with the `inkcre_core` connection and matching JWT + settings. Keep the migration-owner credentials out of both running services. +4. Start the Core Python program with its pinned dependencies, or use its container image, and start + PostgREST. The [Core README](https://github.com/InKCre/core-py#readme) and + [container/runtime guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/docker.md) + own the supported entry points. The Compose stack is a useful topology reference, but its + development credentials and defaults are not a production configuration. +5. Configure HTTPS and reachable service URLs, advertise Core's public address in its Peer + configuration, and arrange process restarts, backups, and upgrades. Verify Core `/readyz` and + authenticated database access before connecting clients. The + [runtime orchestration guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/runtime-orchestration.md) + explains readiness and capability availability. + +## Operate your instance + +A successful first deployment is the beginning of operating the instance. Before relying on it for +important information: + +- Choose hosting that keeps Core available for scheduled collection; sleeping plans cannot promise + continuous collection. +- Keep database backups and verify how you will restore them. Your source accounts and repository + fork are not backups of the info-base. +- Retain deployment credentials securely. Coordinate changes to database passwords and JWT settings + across the services and clients that use them. +- Plan upgrades against the selected Core revision's migration and deployment instructions. Inspect + readiness and collection Jobs after changes; do not treat a running process as proof that + collection succeeded. + +Provider-specific procedures remain in the +[Render deployment guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/render-neon-self-host.md) +and +[Heroku deployment guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/heroku-neon-self-host.md). +The Core runtime documentation linked above owns exact commands and recovery procedures. + +## Continue with collection and use + +Once Core and PostgREST are ready, retain both URLs, the private JWT secret, and the Core Peer +identity. Return to [connecting the CLI](/self-hosted/getting-started#connect-cli), then follow the +same collection, retrieval, and client setup steps as a quick deployment. diff --git a/website/content/en/self-hosted/getting-started.md b/website/content/en/self-hosted/getting-started.md index 8a8a95c..a987c84 100644 --- a/website/content/en/self-hosted/getting-started.md +++ b/website/content/en/self-hosted/getting-started.md @@ -121,44 +121,10 @@ deployment procedure and recovery details live in the ### Manual deployment {#manual-deployment} -You can use your own machine, VPS, containers, or other hosting. No GitHub fork, Neon account, -Render account, or Heroku account is inherently required. These providers automate a portable -runtime consisting of: - -| Component | Responsibility | -| ----------------------------------------------------------- | --------------------------------------------------------------------------------------- | -| PostgreSQL with the required extensions, including pgvector | Persists the info-base and shared configuration. | -| Database initializer | Creates/migrates the schema, provisions runtime roles, and reconciles built-in records. | -| PostgREST | Exposes the admitted database API used by the Web client. | -| Core Python program | Runs collection, Jobs, Extensions, and the Core HTTP API. | - -The initializer is a deployment step, not an additional always-running service. Core and PostgREST -connect to the same initialized database with different runtime roles. The CLI connects to Core; the -Web client uses PostgREST and discovers Core capabilities. The Web app itself can remain at -`app.inkcre.dev` or be hosted separately. - -For a manual installation, work through these steps using the runtime documentation for the Core -revision you selected: - -1. Provision PostgreSQL with the required extensions. Retain an owner connection for initialization - and migrations; do not use that privileged connection as the application's runtime connection. -2. Run Core's ordered database initializer with the runtime profile. It provisions schema and roles - as well as migrations: starting an empty database and only launching Python is not enough. The - [database lifecycle guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/database-contract.md) - owns the initializer commands, credentials, and readiness checks. -3. Configure PostgREST with the `authenticator` connection, the admitted `inkcre` schema, and the - deployment's JWT settings. Configure Core with the `inkcre_core` connection and matching JWT - settings. Keep the migration-owner credentials out of both running services. -4. Start the Core Python program with its pinned dependencies, or use its container image, and start - PostgREST. The [Core README](https://github.com/InKCre/core-py#readme) and - [container/runtime guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/docker.md) - own the supported entry points. The Compose stack is a useful topology reference, but its - development credentials and defaults are not a production configuration. -5. Configure HTTPS and reachable service URLs, advertise Core's public address in its Peer - configuration, and arrange process restarts, backups, and upgrades. Verify Core `/readyz` and - authenticated database access before connecting clients. The - [runtime orchestration guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/runtime-orchestration.md) - explains readiness and capability availability. +For your own database, server, or container setup, follow +[Advanced Self-Hosting](/self-hosted/advanced#manual-deployment). It covers PostgreSQL +initialization, PostgREST, the Core Python program, and operating responsibilities. Return here +after deployment to connect clients and collect your first source. **Checkpoint for every deployment path:** you have a ready Core URL, a PostgREST URL, your private JWT secret, and a persistent Core Peer identity. Continue below; collecting and using information diff --git a/website/content/en/self-hosted/index.md b/website/content/en/self-hosted/index.md index 8325c2c..094df73 100644 --- a/website/content/en/self-hosted/index.md +++ b/website/content/en/self-hosted/index.md @@ -9,13 +9,18 @@ Self-hosting means operating your own InKCre instance: you choose where informat sources it collects, and which clients can access it. You also manage credentials, hosting availability, backups, and updates. This section is for users choosing that responsibility. +For the application-level introduction and access choices, start with +[Getting Started](/getting-started). Here, choose between the step-by-step +[self-hosted walkthrough](/self-hosted/getting-started) and +[Advanced Self-Hosting](/self-hosted/advanced) for manual deployment and operations. + ## Choose a deployment path -| Path | What you manage | Start here | -| ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | -| Fork quick deployment: Neon + Render | Provider accounts and GitHub settings; the workflow initializes the database and deploys Core and PostgREST. | [Render walkthrough](/self-hosted/getting-started#render-neon) | -| Fork quick deployment: Neon + Heroku | The same topology on Heroku, with its billing and runtime settings. | [Heroku walkthrough](/self-hosted/getting-started#heroku-neon) | -| Manual deployment | PostgreSQL initialization, PostgREST, the Core Python process or container, and hosting operations on infrastructure you choose. | [Manual deployment overview](/self-hosted/getting-started#manual-deployment) | +| Path | What you manage | Start here | +| ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | +| Fork quick deployment: Neon + Render | Provider accounts and GitHub settings; the workflow initializes the database and deploys Core and PostgREST. | [Render walkthrough](/self-hosted/getting-started#render-neon) | +| Fork quick deployment: Neon + Heroku | The same topology on Heroku, with its billing and runtime settings. | [Heroku walkthrough](/self-hosted/getting-started#heroku-neon) | +| Manual deployment | PostgreSQL initialization, PostgREST, the Core Python process or container, and hosting operations on infrastructure you choose. | [Advanced](/self-hosted/advanced#manual-deployment) | Neon, Render, and Heroku are convenient deployment options, not product dependencies. Forking provides a ready-made deployment workflow; it is not required to run InKCre. Both quick deployments From baf8c03497b2a48146e5f904787ba98b7065482f Mon Sep 17 00:00:00 2001 From: Lan_zhijiang Date: Sun, 20 Sep 2026 16:10:24 +0800 Subject: [PATCH 05/18] =?UTF-8?q?docs(guides):=20=E6=8B=86=E5=88=86?= =?UTF-8?q?=E5=8F=AF=E5=A4=8D=E7=94=A8=E7=9A=84=E7=94=A8=E6=88=B7=E6=93=8D?= =?UTF-8?q?=E4=BD=9C=E4=B8=8E=E9=83=A8=E7=BD=B2=E6=8C=87=E5=8D=97?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 自部署 Getting Started 仅编排阅读路径 - 独立提供连接、采集、检索、定时任务、来源、日常使用及排障指南 - Render 和 Heroku 部署分别成页，跨文档复用改用页面链接 --- website/.vitepress/config.mts | 14 + website/README.md | 10 +- website/content/en/getting-started.md | 12 +- website/content/en/guide/connect-cli.md | 62 ++ website/content/en/guide/daily-use.md | 80 +++ website/content/en/guide/first-source.md | 76 +++ website/content/en/guide/schedules.md | 45 ++ website/content/en/guide/search.md | 42 ++ website/content/en/guide/sources.md | 145 +++++ website/content/en/guide/troubleshooting.md | 25 + website/content/en/self-hosted/advanced.md | 5 +- .../content/en/self-hosted/getting-started.md | 563 +----------------- website/content/en/self-hosted/heroku-neon.md | 56 ++ website/content/en/self-hosted/index.md | 13 +- website/content/en/self-hosted/render-neon.md | 58 ++ 15 files changed, 656 insertions(+), 550 deletions(-) create mode 100644 website/content/en/guide/connect-cli.md create mode 100644 website/content/en/guide/daily-use.md create mode 100644 website/content/en/guide/first-source.md create mode 100644 website/content/en/guide/schedules.md create mode 100644 website/content/en/guide/search.md create mode 100644 website/content/en/guide/sources.md create mode 100644 website/content/en/guide/troubleshooting.md create mode 100644 website/content/en/self-hosted/heroku-neon.md create mode 100644 website/content/en/self-hosted/render-neon.md diff --git a/website/.vitepress/config.mts b/website/.vitepress/config.mts index 9af7b23..545c138 100644 --- a/website/.vitepress/config.mts +++ b/website/.vitepress/config.mts @@ -57,11 +57,25 @@ export default defineConfig({ sidebar: { '/': [ { text: 'Getting Started', link: '/getting-started' }, + { + text: 'User Guide', + items: [ + { text: 'Connect the CLI', link: '/guide/connect-cli' }, + { text: 'Collect Your First Source', link: '/guide/first-source' }, + { text: 'Find What You Saved', link: '/guide/search' }, + { text: 'Schedule Collection and Indexing', link: '/guide/schedules' }, + { text: 'Connect More Sources', link: '/guide/sources' }, + { text: 'Use Your Information', link: '/guide/daily-use' }, + { text: 'Troubleshooting', link: '/guide/troubleshooting' }, + ], + }, { text: 'Self-Hosted', items: [ { text: 'Overview', link: '/self-hosted/' }, { text: 'Getting Started', link: '/self-hosted/getting-started' }, + { text: 'Render and Neon', link: '/self-hosted/render-neon' }, + { text: 'Heroku and Neon', link: '/self-hosted/heroku-neon' }, { text: 'Advanced', link: '/self-hosted/advanced' }, ], }, diff --git a/website/README.md b/website/README.md index 96ba509..36083e3 100644 --- a/website/README.md +++ b/website/README.md @@ -29,9 +29,13 @@ pnpm --dir website audit --audit-level high - Future Chinese source will live under `content/zh/` and be published under `/zh/`. - Only English is active until the Chinese route set is complete or the locale switch has a deliberate fallback. -- The current routes are `/`, `/getting-started`, `/self-hosted/`, `/self-hosted/getting-started`, - `/self-hosted/advanced`, `/developer/`, `/developer/architecture`, `/developer/contributing`, and - `/about/`. +- `/getting-started` owns the application-level What, Why, and How introduction. +- `/self-hosted/` contains its Getting Started path, Render/Heroku quick-deployment guides, and + Advanced guide. Its Getting Started page orders the journey rather than duplicating procedures. +- `/guide/` leaf pages own reusable client, collection, retrieval, scheduling, source, daily-use, + and troubleshooting procedures. Link to these pages from any onboarding path rather than to + sections buried inside the self-hosted walkthrough. +- `/developer/` contains architecture and contribution guidance; `/about/` describes the project. - Section indexes use trailing-slash routes, such as `/developer/`. - Leaf pages use lowercase ASCII kebab-case routes without an extension, such as `/developer/architecture`. diff --git a/website/content/en/getting-started.md b/website/content/en/getting-started.md index 7b37de0..a3bb8ac 100644 --- a/website/content/en/getting-started.md +++ b/website/content/en/getting-started.md @@ -51,9 +51,9 @@ does not create an instance for you. first collection. [Advanced](/self-hosted/advanced) covers manual deployment and operating it on infrastructure you choose. - **Already have access to an instance:** obtain connection details from the person operating it, - then follow the [CLI connection steps](/self-hosted/getting-started#connect-cli) and subsequent - collection and client setup sections. Skip the deployment steps. Only connect information and - tools you are authorized to use with that instance. + then follow [Connect the CLI](/guide/connect-cli), [collect a source](/guide/first-source), and + [connect your everyday tools](/guide/daily-use). Skip the deployment steps. Only connect + information and tools you are authorized to use with that instance. This guide does not assume a hosted sign-up service or separate private user accounts inside an instance. Access arrangements and trust matter; do not treat a shared instance as an isolated @@ -68,9 +68,9 @@ each requires. ### 3. Find and use what you collected -Maintain the search index, search for something you remember, and open a result. Then connect the -Web app or a tool you already use so that the collection is accessible in your daily work. The -walkthrough covers these steps after deployment; they apply to an existing compatible instance too. +Follow [Find What You Saved](/guide/search) to maintain the search index, search for something you +remember, and open a result. Then [Use Your Information](/guide/daily-use) to connect the Web app or +a tool you already use. These guides work independently of your deployment choice. AI organization and semantic retrieval are optional next steps with their own provider and maintenance setup. InKCre does not automatically configure a daily digest or Telegram/email push diff --git a/website/content/en/guide/connect-cli.md b/website/content/en/guide/connect-cli.md new file mode 100644 index 0000000..c849a39 --- /dev/null +++ b/website/content/en/guide/connect-cli.md @@ -0,0 +1,62 @@ +--- +title: Connect the CLI +description: Connect the command-line tool to an existing InKCre instance. +--- + +# Connect the CLI + +You need a ready Core URL and the instance's private JWT secret. Obtain them from your deployment or +its operator. Keep the secret private; it grants instance authority, not an isolated personal login. + +The CLI handles setup operations that do not yet have a Web form. It connects over HTTPS; it does +not run another server on your computer. + +1. Install [Python](https://www.python.org/downloads/) 3.12 or later if needed. Check + `python --version` in a terminal; use `python3` if that is your system's command name. +2. Create a local environment: + + ```sh + python -m venv .venv + ``` + + Activate it with `source .venv/bin/activate` on macOS/Linux, or `.venv\Scripts\Activate.ps1` in + Windows PowerShell. Then install the published CLI: + + ```sh + python -m pip install inkcre-cli + inkcre-cli --help + ``` + +3. In a local folder outside any Git repository, use a text editor to create `connection.json`: + + ```json + { + "base_url": "https://YOUR-CORE-HOST", + "jwt_secret": "YOUR-SAVED-JWT-SECRET" + } + ``` + + Replace both values, preserving the quotes. Use the Core base URL without `/readyz`. This file + contains a credential; keep it private. + +4. From that folder, save and check the connection: + + ```sh + inkcre-cli connection set personal --input connection.json + inkcre-cli connection use personal + inkcre-cli connection check + ``` + + Both readiness and the authenticated read should succeed. You can remove the temporary + `connection.json` afterward: the CLI retains it at `.inkcre/cli/connections.json` in your home + directory. Protect that file too. + +**Checkpoint:** `connection check` can read your instance. For `401`, check the secret; for a +connection failure, check the Core URL and wake `/readyz`. In later terminal sessions, reactivate +the environment before using `inkcre-cli`. + +The [CLI reference](https://github.com/InKCre/core-py/blob/main/cli/README.md) owns command details. +`--help` explains a command; `--schema` on input-taking commands shows the configuration accepted by +your running instance. + +Next: [Collect your first source](/guide/first-source). diff --git a/website/content/en/guide/daily-use.md b/website/content/en/guide/daily-use.md new file mode 100644 index 0000000..5b57b38 --- /dev/null +++ b/website/content/en/guide/daily-use.md @@ -0,0 +1,80 @@ +--- +title: Use Your Information +description: Connect the Web app and everyday tools, with optional AI organization. +--- + +# Use Your Information + +Start with a working instance and [a successful search](/guide/search). Retain your Core URL, +PostgREST URL, and private JWT secret. If someone else operates the instance, ask them to register +your browser Peer rather than assuming database access. + +## Read and explore in the Web app + +The browser needs its **own** Client ID. Do not reuse the Core Peer ID: the browser excludes its own +identity when looking for another Peer to execute a capability. Settings does not yet create this +record for you. + +1. In your dedicated Neon project's **SQL Editor**, select the same default branch and `neondb` + database as deployment. For a manually hosted database, use your PostgreSQL SQL client against + the initialized InKCre database instead. Run this once to register your browser: + + ```sql + INSERT INTO inkcre.peers (id, name, config) + VALUES ( + gen_random_uuid(), + 'My web browser', + '{"extension_registry_url":"https://registry.inkcre.dev"}'::jsonb + ) + RETURNING id; + ``` + + Save the returned UUID as your browser Client ID. This adds one record without replacing Core's + record. Reuse it when reconnecting this browser. + +2. Wake Core `/readyz`, then open [Web app Settings](https://app.inkcre.dev/settings). +3. Enter **PostgreSQL REST URL** = PostgREST base URL, **JWT Secret** = your saved secret, and + **Client ID** = the new browser UUID. Choose **Save**, then reload. The Clients list should + include Core, reporting online. +4. Open **Info Base**, search for the phrase that worked in the [search guide](/guide/search), and + inspect a result. Use **View content** for supported content and the graph view to follow + relationships. The list is a search surface, not every stored Block. Some source renderers need a + compatible Web Extension; the CLI's `get_text` remains a way to read Core-resolved text. +5. Bookmark the app. Another browser/device needs its own connection setup. **Export** omits the + secret and is not an info-base backup. + +## Use your information from a terminal or AI tool + +Keep `inkcre-cli recall 'your clue' --mode lexical` available wherever you work. A terminal-based +assistant you trust can use the installed CLI and named connection. For example: + +> Use my InKCre personal connection to find articles I saved about distributed systems. Read the +> matches and cite their original links. Do not change my sources or data. + +The connection has owner authority; only give it to a tool you trust. Its model provider may receive +the content it reads. + +For clients that connect over MCP instead of running terminal commands, follow the +[MCP Sink setup](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/mcp-sink.md). Create +and enable a `core.mcp.v1` Sink with a separate PAT, then configure Bearer authentication for +`https://YOUR-CORE-HOST/sinks/SINK_ID/mcp`. This needs explicit Sink and client setup; the CLI has +no `sink` command. The MCP PAT is not `JWT_SECRET`. Check the client's authentication support or the +documented tunnel before choosing this route. + +These paths make your saved information available on demand. They do not configure proactive +notifications or a daily briefing. + +## Add AI organization when you need it + +Once collection and reading work, configure a model provider and Agent for rumination, then +explicitly reconsider a Block. Follow the +[Core organization configuration](https://github.com/InKCre/core-py/blob/main/docs/30-unit-tdd/organization.md) +for the Agent and deployment settings before using **Ruminate** or +`inkcre-cli organization ruminate BLOCK_ID`. + +Inspect the Job and graph afterward. A valid result may add nothing. Re-index after new content is +added to find it by words. Semantic search additionally needs an embedding provider/profile and +maintenance; an LLM API key alone does not enable it. Selected content leaves your instance for +configured AI providers and may incur charges. + +Next: [Troubleshooting](/guide/troubleshooting). diff --git a/website/content/en/guide/first-source.md b/website/content/en/guide/first-source.md new file mode 100644 index 0000000..ba859e4 --- /dev/null +++ b/website/content/en/guide/first-source.md @@ -0,0 +1,76 @@ +--- +title: Collect Your First Source +description: Install the RSS collector and collect your first feed. +--- + +# Collect Your First Source + +Before you start, [connect the CLI](/guide/connect-cli) to your instance. + +Choose a publication you already read and copy its **RSS or Atom feed URL**. Its normal homepage URL +is not usually a feed URL. Look for an RSS/Subscribe link on the publication. + +## Enable the collector + +Install the RSS Extension on Core, then enable it: + +```sh +inkcre-cli extension install inkcre/rss --version 0.2.0 +inkcre-cli extension enable inkcre/rss +inkcre-cli source types +``` + +Version `0.2.0` is a published Python release for Core Host `0.2.x`. For a different Host version, +check the [RSS release listing](https://registry.inkcre.dev/v1/extensions/inkcre/rss) for a +published release with a compatible `python.host_sdk_version`. Do not guess a version or use +`latest`. If already installed, inspect `inkcre-cli extension get inkcre/rss` before changing it. + +The type list should contain `extensions.rss.rss.Source` and `extensions.rss.atom.Source`. The Web +app's Extensions switch controls its browser runtime; it does not enable this Core collector. + +## Add and run the source + +1. Create `feed.json` in your local working folder: + + ```json + { + "nickname": "My first feed", + "config": { + "feed_url": "https://YOUR-PUBLICATION/FEED", + "fetch_full_text": false, + "download_enclosures": false + } + } + ``` + + Replace the URL. These first-run settings collect feed content without extra article fetches or + attachment downloads. + +2. Create the Source, choosing the matching feed format: + + ```sh + inkcre-cli source create --type extensions.rss.rss.Source --input feed.json + ``` + + For Atom, use `extensions.rss.atom.Source`. Record the returned Source `id`. + +3. Collect it. Replace `42` with your Source ID: + + ```sh + inkcre-cli source collect 42 --input-json '{}' + ``` + +4. The response contains a **Job ID**, different from the Source ID. Replace `17` with it: + + ```sh + inkcre-cli job wait 17 --for 30s + ``` + + Look for `status: finished`. If still `pending` or `running`, observe again rather than creating + another collect job. If `failed`, inspect `inkcre-cli job get 17 --json` and its diagnostics. + Ending observation does not stop the Job. + +**Checkpoint:** the collection Job finished. A feed exposes only what its publisher currently +provides; success does not mean its entire historical archive was imported. + +Next: [Find what you saved](/guide/search). diff --git a/website/content/en/guide/schedules.md b/website/content/en/guide/schedules.md new file mode 100644 index 0000000..45d0dc7 --- /dev/null +++ b/website/content/en/guide/schedules.md @@ -0,0 +1,45 @@ +--- +title: Schedule Collection and Indexing +description: Keep sources and search indexes current with explicit schedules. +--- + +# Schedule Collection and Indexing + +First complete [one collection](/guide/first-source) and [a successful search](/guide/search). Keep +the Source ID returned when you created the Source. + +After the manual run works, create `collect-hourly.json`, replacing `42` with your Source ID: + +```json +{ + "schedule": "0 * * * *", + "job_parameters": { "source": 42, "config": {} } +} +``` + +```sh +inkcre-cli cron create --job-type core.source.collect.v1 --input collect-hourly.json +``` + +This five-field schedule means “at minute zero of every hour.” Add independent index maintenance so +later items become searchable. Save this as `index-periodically.json`: + +```json +{ + "schedule": "*/10 * * * *", + "job_parameters": {} +} +``` + +```sh +inkcre-cli cron create --job-type core.feature_retrieval.lexical.maintain.v1 --input index-periodically.json +inkcre-cli cron list +``` + +Create each schedule once. Inspect its returned Cron ID with `inkcre-cli cron get ID`; `last_job` +identifies the Job to check. Pause it with `inkcre-cli cron disable ID`. Replace `ID` with the +actual number. Your terminal may be closed, but Core must run. Sleeping hosts miss occurrences and +do not automatically catch up. Independent indexing also means new items may not become searchable +immediately. + +Next: [Add more sources](/guide/sources). diff --git a/website/content/en/guide/search.md b/website/content/en/guide/search.md new file mode 100644 index 0000000..412bcaa --- /dev/null +++ b/website/content/en/guide/search.md @@ -0,0 +1,42 @@ +--- +title: Find What You Saved +description: Index collected information and retrieve it by remembered words. +--- + +# Find What You Saved + +You need a connected CLI and at least one completed collection. If you have not collected anything +yet, start with [your first source](/guide/first-source). + +Collection and indexing are separate. Create a lexical-index maintenance Job after collection to +enable search by remembered words, without an AI provider: + +```sh +inkcre-cli job create --type core.feature_retrieval.lexical.maintain.v1 --input-json '{"parameters":{}}' +``` + +Use its returned Job ID with `inkcre-cli job wait JOB_ID --for 30s`, replacing `JOB_ID` with the +number. Check its status and `state` for indexed records and diagnostics. Maintenance processes a +bounded batch; repeat if you imported more than one batch can index. + +Copy a distinctive phrase from an item in your feed and search: + +```sh +inkcre-cli recall 'A phrase from your feed item' --mode lexical +``` + +Read a returned Block's content and relationships, replacing `123` with that Block's ID: + +```sh +inkcre-cli resolver invoke block:123 --method get_text +inkcre-cli graph neighborhood block:123 +``` + +**Checkpoint:** you can retrieve a real item you recognize and see its source relationships. You +have completed the first loop: source → saved information → information you can use. + +This first form of organization comes from the source: feed items belong to feeds, GitHub +repositories can belong to Lists, and mail has sender and mailbox relationships. It does not mean +InKCre has already summarized, tagged, or reorganized everything with AI. + +Next: [Keep collecting and searching](/guide/schedules). diff --git a/website/content/en/guide/sources.md b/website/content/en/guide/sources.md new file mode 100644 index 0000000..75145b1 --- /dev/null +++ b/website/content/en/guide/sources.md @@ -0,0 +1,145 @@ +--- +title: Connect More Sources +description: Connect GitHub, email, Telegram, and other supported sources. +--- + +# Connect More Sources + +These instructions use an already [connected CLI](/guide/connect-cli). Follow the +[first-source walkthrough](/guide/first-source) once to learn how Source IDs and collection Jobs +work. + +For each new source: enable its Core Extension, create the Source, collect once, inspect the Job, +run index maintenance, and search for a known item. Only then add a schedule. + +The versions below are published releases for Core Host `0.2.x`. Check the linked release listings +and your running type's `--schema` when versions differ. Source files and command output may contain +account credentials; do not publish them. + +## GitHub Stars and Lists + +1. Create a personal access token for the account whose Stars and Lists you want to collect, + following + [GitHub's token instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens). + Its account and permissions determine which data is visible. +2. Enable the Extension: + + ```sh + inkcre-cli extension install inkcre/github --version 0.3.0 + inkcre-cli extension enable inkcre/github + ``` + +3. Save `github.json` with your token: + + ```json + { + "nickname": "My GitHub saves", + "config": { "github_token": "YOUR-GITHUB-TOKEN" } + } + ``` + + ```sh + inkcre-cli source create --type extensions.github.stars.Source --input github.json + ``` + +4. Collect the returned Source ID using the [first-source procedure](/guide/first-source). After + indexing, search for a repository you starred. Each run synchronizes current Stars and Lists; it + is not a code backup or a full GitHub activity archive. + +See the [GitHub collector](https://github.com/InKCre/core-py/blob/main/extensions/github/README.md) +and [published releases](https://registry.inkcre.dev/v1/extensions/inkcre/github). + +## Email over IMAP + +1. Find your provider's IMAP hostname and enable IMAP if required. Obtain an app-specific password + where supported. Accounts requiring an unsupported authentication method cannot be connected by + substituting an ordinary login password. +2. Install and enable `inkcre/mail` version `0.3.0`, using the two Extension commands above with + `inkcre/mail` in place of `inkcre/github`. +3. Save `mail.json`, replacing the host and credentials: + + ```json + { + "nickname": "My mail", + "config": { + "protocol": "imap", + "parameters": { + "host": "YOUR-IMAP-HOST", + "port": 993, + "security": "tls", + "username": "YOUR-MAIL-LOGIN", + "password": "YOUR-APP-PASSWORD" + }, + "ordinary_mark_as_seen": false, + "synchronize_deletions": false + } + } + ``` + + ```sh + inkcre-cli source create --type extensions.mail.source.Source --input mail.json + ``` + +4. Send yourself a test email **after creating the Source**, then collect its ID. Ordinary + collection starts with new mail; an empty first run does not imply login failed. Inspect mailbox + diagnostics even if the Job finishes. This example preserves unread state; the collector's + default would mark ordinary collected mail as seen. +5. For older mail, request a small explicit backfill. Replace `42` with the Mail Source ID and + choose dates for your mailbox: + + ```sh + inkcre-cli source backfill 42 --input-json '{"since":"2026-09-01","before":"2026-09-08"}' + ``` + + The start date is included and the end date excluded. Observe the Job and index afterward. This + collector does not send mail or create email digests. + +See the +[Mail configuration](https://github.com/InKCre/core-py/blob/main/docs/30-unit-tdd/mail-extension.md) +and [published releases](https://registry.inkcre.dev/v1/extensions/inkcre/mail). + +## Send or forward messages from Telegram + +1. Create a dedicated bot with [@BotFather](https://t.me/botfather): send `/newbot`, follow its + naming prompts, and retain the token. Obtain your own numeric user ID using + [Telegram Desktop's data export](https://telegram.org/blog/export-and-more): open **Settings → + Advanced → Export Telegram data**, include personal information, and choose JSON format. In the + exported JSON, use `personal_information.user_id`, not your `@username` or the bot's ID. Telegram + may require confirmation from another device or a waiting period. +2. Install and enable `inkcre/telegram` version `0.3.0` with the same Extension commands. +3. Save `telegram.json`, replacing the token and example user ID: + + ```json + { + "nickname": "My Telegram inbox", + "config": { + "bot_token": "YOUR-BOT-TOKEN", + "bound_user_id": 123456789, + "download_attachments": false + } + } + ``` + + ```sh + inkcre-cli source create --type extensions.telegram.source.Source --input telegram.json + ``` + +4. Send a distinctive text message in a private chat with the bot, then collect the Source ID. + Successfully saved messages receive a 👍 reaction. After indexing, search for its text. +5. Add a schedule to forward messages without running the CLI each time. `*/5 * * * *` polls every + five minutes while Core is awake. Telegram retains updates for a limited time, so a long-sleeping + instance should not be your only copy. + +Use one bot for one Source. This is a capture inbox, not group/channel history import or a +notification destination. Attachments are metadata-only in this example; enable +`download_attachments` when you want their bytes saved too. See the +[Telegram collector](https://github.com/InKCre/core-py/blob/main/extensions/telegram/README.md) and +[published releases](https://registry.inkcre.dev/v1/extensions/inkcre/telegram). + +Add more RSS/Atom subscriptions with one Source per feed. Memos-compatible capture is also available +through a separately configured +[Memos Extension](https://github.com/InKCre/core-py/blob/main/docs/30-unit-tdd/memos-extension.md). +An arbitrary service or browser account is not automatically a supported Source; check for a +collector before assuming its history can be imported. + +Next: [Use your information](/guide/daily-use). diff --git a/website/content/en/guide/troubleshooting.md b/website/content/en/guide/troubleshooting.md new file mode 100644 index 0000000..d51f5bd --- /dev/null +++ b/website/content/en/guide/troubleshooting.md @@ -0,0 +1,25 @@ +--- +title: Troubleshooting +description: Check deployment, connection, collection, and retrieval problems. +--- + +# Troubleshooting + +Use the symptom below to find the next check. Keep credentials out of logs or screenshots you share. + +| What you observe | What to check next | +| ---------------------------------------- | --------------------------------------------------------------------------------------------------------- | +| Deployment workflow missing | Enable Actions in your fork and check that it is up to date with upstream `main`. | +| Core slow or offline | Open Core `/readyz`; inspect host logs if it stays unready. PostgREST may sleep separately. | +| Client returns `401` | Check endpoint and secret. Anonymous PostgREST `401` is expected; authenticated `401` is not. | +| Source type absent | Install a compatible Extension and enable it on Core, not only in the Web app. | +| Job stays pending | Confirm Core is awake and its collector enabled. Job creation only confirms acceptance. | +| Collection finished but search empty | Confirm the source exposed the item; run lexical maintenance and inspect diagnostics. | +| CLI works but Web cannot find a provider | Check Core readiness and that browser and Core Peer IDs differ. | +| Old mail missing | Ordinary collection starts with new mail. Request a historical backfill. | +| Truncated content or no attachment bytes | First-run examples avoid extra downloads. Check the collector's settings. | +| Request outcome uncertain | Read the Job or resulting data before repeating a write; a lost response does not prove nothing happened. | + +Keep hosting and connection credentials in your password manager. Review your database's +backup/restore options and hosting usage before collecting irreplaceable information. A fork, a +client configuration export, and a source account are not backups of your info-base. diff --git a/website/content/en/self-hosted/advanced.md b/website/content/en/self-hosted/advanced.md index 7f2a59b..cec8c35 100644 --- a/website/content/en/self-hosted/advanced.md +++ b/website/content/en/self-hosted/advanced.md @@ -75,5 +75,6 @@ The Core runtime documentation linked above owns exact commands and recovery pro ## Continue with collection and use Once Core and PostgREST are ready, retain both URLs, the private JWT secret, and the Core Peer -identity. Return to [connecting the CLI](/self-hosted/getting-started#connect-cli), then follow the -same collection, retrieval, and client setup steps as a quick deployment. +identity. Continue with [Connect the CLI](/guide/connect-cli), +[your first source](/guide/first-source), [search](/guide/search), and +[everyday tools](/guide/daily-use). These shared guides apply to manual and quick deployments alike. diff --git a/website/content/en/self-hosted/getting-started.md b/website/content/en/self-hosted/getting-started.md index a987c84..a9be917 100644 --- a/website/content/en/self-hosted/getting-started.md +++ b/website/content/en/self-hosted/getting-started.md @@ -1,550 +1,49 @@ --- title: Getting Started -description: - Create your own InKCre instance, collect your first feed, connect personal sources, and find your - information from the web or your everyday tools. +description: Follow the self-hosted path from deployment to collecting and using your information. --- -# Getting Started +# Self-Hosted Getting Started -This is the [Self-Hosted](/self-hosted/) walkthrough: you will operate your own InKCre instance. It -is not a requirement that every InKCre user deploy a server. +This is the [Self-Hosted](/self-hosted/) path through InKCre: you will operate your own instance. +For the application's What, Why, and How, begin with [Getting Started](/getting-started). -You have information scattered across subscriptions, saved repositories, messages, and email. This -guide takes you from **no InKCre instance** to collecting a source you care about, finding something -you collected, and making that collection available in your daily workflow. +You will use a browser, a text editor, and a few terminal commands. No InKCre development setup or +AI model is needed for the first collection-and-search journey. Each guide below is self-contained +and can also be used with an existing compatible instance. -Start with one RSS feed. Once it works, add your personal sources one at a time. You do not need to -develop InKCre or configure an AI model for this first journey. The fork-based deployment paths also -avoid running Docker locally. You will use a browser, a text editor, and a few terminal commands. +## 1. Deploy an instance -InKCre is under active development. Some setup still uses its command-line tool rather than a setup -wizard. Collection preserves source relationships; further AI organization needs its own -configuration. There is no default daily digest or automatic Telegram/email notification service. +Choose **one** deployment guide: -## 1. Create your instance +- [Render and Neon](/self-hosted/render-neon): fork-based quick deployment on Render. +- [Heroku and Neon](/self-hosted/heroku-neon): the equivalent quick deployment on Heroku. +- [Advanced Self-Hosting](/self-hosted/advanced): initialize PostgreSQL, configure PostgREST, and + run Core on infrastructure you choose. -Your instance stores your information and runs collection. The public -[Web app](https://app.inkcre.dev/settings) is a client you connect to it; opening the app does not -create a private instance for you. +Neon and the compute providers are convenience options, not requirements. At the end, retain a ready +Core URL, a PostgREST URL, your private JWT secret, and the Core Peer ID. -Choose one deployment path. **Neon + Render and Neon + Heroku are alternative fork-based quick -deployments**, not requirements of InKCre itself. Both workflows initialize the database and run -Core and PostgREST for you. If you prefer your own server or database, follow -[manual deployment](#manual-deployment) instead, then return to step 2. +## 2. Complete the first collection-and-search loop -For either quick deployment, use GitHub, Neon, and one compute provider: +Follow these shared user guides in order. If you already have an instance, start here. -| Account | What it does | -| ---------------- | --------------------------------------------------------------------------------------------------------------- | -| GitHub | Keeps your fork of InKCre and runs its deployment workflow. A fork is your own repository copy. | -| Neon | Stores your info-base in a PostgreSQL database. | -| Render or Heroku | Runs Core, which collects and processes information, and PostgREST, which connects the Web app to the database. | +1. [Connect the CLI](/guide/connect-cli) and verify authenticated access. +2. [Collect your first source](/guide/first-source), using one public RSS or Atom feed. +3. [Find what you saved](/guide/search): maintain the lexical index, search, and inspect a result. -Use a dedicated Neon project for this instance and a password manager to retain credentials. An API -key lets the deployment workflow act on your hosting account; it is different from your sign-in -password. +**Checkpoint:** you can retrieve a real item from your source. Collection and indexing are separate; +neither requires you to set up AI organization first. -### Quick deployment: Render and Neon {#render-neon} +## 3. Make it useful day to day -1. Open [InKCre/core-py](https://github.com/InKCre/core-py), choose **Fork**, and create your - repository copy. Open its **Actions** tab and enable workflows if GitHub asks. -2. Create a project in [Neon](https://console.neon.tech). Keep the default `neondb` database and - `neondb_owner` role. Save the project ID and create a Neon API key with access to it. -3. Create a workspace in [Render](https://dashboard.render.com). Save its workspace ID from Settings - and create a Render API key. Your fork must be public, or Render must already have permission to - read it. -4. In your fork, open **Settings → Secrets and variables → Actions**. Use **New repository secret** - under **Secrets** and **New repository variable** under **Variables** to add: +- [Schedule collection and indexing](/guide/schedules) after the manual steps work. +- [Connect more sources](/guide/sources), such as GitHub saves, email, and Telegram messages. +- [Use your information](/guide/daily-use) from the Web app, a terminal, or a trusted assistant. + This also explains optional AI organization and MCP access. -| Kind | Name | Value | -| -------- | ----------------------- | ------------------------------------------------------------------------------------ | -| Secret | `NEON_API_KEY` | Your Neon API key. | -| Secret | `RENDER_API_KEY` | Your Render API key. | -| Secret | `JWT_SECRET` | A new random secret of at least 32 ASCII characters, saved in your password manager. | -| Variable | `NEON_PROJECT_ID` | Your Neon project ID. | -| Variable | `RENDER_OWNER_ID` | Your Render workspace ID. | -| Variable | `RENDER_SERVICE_PREFIX` | A unique lowercase prefix such as `alex-inkcre`, between 3 and 40 characters. | +These paths make information available on demand. They do not configure automatic daily digests or +Telegram/email push notifications. Hosting that sleeps cannot guarantee continuous collection. -5. Open **Actions → Deploy self-hosted InKCre → Run workflow**, select your fork's `main` branch, - and run it. Wait for completion. If it fails, open the failed step, correct the reported problem, - and rerun using the same credentials. -6. Open the completed run's summary. Save the **Core URL**, **PostgREST URL**, and **Core Peer ID**. - The workflow deliberately does not print your secret; retain the original value. -7. Open the Core URL with `/readyz` appended. Continue when it returns HTTP `200`. A sleeping - service may take time to start. - -**Checkpoint:** deployment succeeded, and you have both service URLs and your `JWT_SECRET`. Use the -**Core URL** for the CLI and the **PostgREST URL** for the Web app later. - -The workflow selects Render Free services. They sleep when idle, share account usage limits, and -cannot guarantee continuous collection. Check -[Render's current limits](https://render.com/docs/free) and your Neon plan before relying on them. -Continuous scheduled collection needs hosting that keeps Core running. The maintained deployment -procedure and recovery details live in the -[Core self-hosting guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/render-neon-self-host.md). - -### Quick deployment: Heroku and Neon {#heroku-neon} - -Choose this instead of the Render steps; you do not need both providers. The database still lives in -Neon, while Heroku runs two apps: Core and PostgREST. - -1. Fork `core-py` and enable Actions. Create a dedicated Neon project, keeping `neondb` and - `neondb_owner`, and retain the project ID and API key as described above. -2. Create a [Heroku account](https://dashboard.heroku.com/) with billing enabled and obtain an API - key. The workflow creates the two apps; you do not need to provision a Heroku database add-on. -3. In your fork's **Settings → Secrets and variables → Actions**, add the following settings. - Generate independent random values for the three credential secrets and retain them in your - password manager. - -| Kind | Name | Value | -| -------- | ----------------------------- | ---------------------------------------------------------------------------------- | -| Secret | `NEON_API_KEY` | Your Neon API key. | -| Secret | `HEROKU_API_KEY` | Your Heroku API key. | -| Secret | `JWT_SECRET` | A new random secret of at least 32 ASCII characters. | -| Secret | `CORE_DATABASE_PASSWORD` | A separate random password of at least 32 ASCII characters. | -| Secret | `POSTGREST_DATABASE_PASSWORD` | Another random password of at least 32 ASCII characters. | -| Variable | `NEON_PROJECT_ID` | Your Neon project ID. | -| Variable | `HEROKU_APP_PREFIX` | A unique lowercase app prefix, such as `alex-inkcre`, between 3 and 18 characters. | - -4. Open **Actions → Deploy self-hosted InKCre to Heroku → Run workflow**, select your fork's `main` - branch, and wait for the run to succeed. -5. Save the Core URL, PostgREST URL, and Core Peer ID from its summary. Open Core `/readyz` and wait - for HTTP `200`, then continue to step 2 below. - -The workflow runs one Eco web dyno per app. Heroku charges and -[Eco sleep behavior](https://devcenter.heroku.com/articles/eco-dyno-hours) apply. Keep the same -database passwords on reruns; replacing them is a coordinated credential rotation, not a routine -redeploy. As with Render Free, sleeping Core cannot provide continuous collection. The maintained -deployment procedure and recovery details live in the -[Core Heroku self-hosting guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/heroku-neon-self-host.md). - -### Manual deployment {#manual-deployment} - -For your own database, server, or container setup, follow -[Advanced Self-Hosting](/self-hosted/advanced#manual-deployment). It covers PostgreSQL -initialization, PostgREST, the Core Python program, and operating responsibilities. Return here -after deployment to connect clients and collect your first source. - -**Checkpoint for every deployment path:** you have a ready Core URL, a PostgREST URL, your private -JWT secret, and a persistent Core Peer identity. Continue below; collecting and using information -works the same way regardless of the hosting provider. - -**Keep your instance credentials private.** `JWT_SECRET` grants control of the instance, not a -limited personal login. Do not paste it into an online JWT generator, an issue, a chat, or your -public fork. This deployment does not isolate different users from one another. Start with your own -information and trusted devices. - -## 2. Connect the command-line tool {#connect-cli} - -The CLI handles setup operations that do not yet have a Web form. It connects over HTTPS; it does -not run another server on your computer. - -1. Install [Python](https://www.python.org/downloads/) 3.12 or later if needed. Check - `python --version` in a terminal; use `python3` if that is your system's command name. -2. Create a local environment: - - ```sh - python -m venv .venv - ``` - - Activate it with `source .venv/bin/activate` on macOS/Linux, or `.venv\Scripts\Activate.ps1` in - Windows PowerShell. Then install the published CLI: - - ```sh - python -m pip install inkcre-cli - inkcre-cli --help - ``` - -3. In a local folder outside any Git repository, use a text editor to create `connection.json`: - - ```json - { - "base_url": "https://YOUR-CORE-HOST", - "jwt_secret": "YOUR-SAVED-JWT-SECRET" - } - ``` - - Replace both values, preserving the quotes. Use the Core base URL without `/readyz`. This file - contains a credential; keep it private. - -4. From that folder, save and check the connection: - - ```sh - inkcre-cli connection set personal --input connection.json - inkcre-cli connection use personal - inkcre-cli connection check - ``` - - Both readiness and the authenticated read should succeed. You can remove the temporary - `connection.json` afterward: the CLI retains it at `.inkcre/cli/connections.json` in your home - directory. Protect that file too. - -**Checkpoint:** `connection check` can read your instance. For `401`, check the secret; for a -connection failure, check the Core URL and wake `/readyz`. In later terminal sessions, reactivate -the environment before using `inkcre-cli`. - -The [CLI reference](https://github.com/InKCre/core-py/blob/main/cli/README.md) owns command details. -`--help` explains a command; `--schema` on input-taking commands shows the configuration accepted by -your running instance. - -## 3. Collect your first RSS feed - -Choose a publication you already read and copy its **RSS or Atom feed URL**. Its normal homepage URL -is not usually a feed URL. Look for an RSS/Subscribe link on the publication. - -### Enable the collector - -Install the RSS Extension on Core, then enable it: - -```sh -inkcre-cli extension install inkcre/rss --version 0.2.0 -inkcre-cli extension enable inkcre/rss -inkcre-cli source types -``` - -Version `0.2.0` is a published Python release for Core Host `0.2.x`. For a different Host version, -check the [RSS release listing](https://registry.inkcre.dev/v1/extensions/inkcre/rss) for a -published release with a compatible `python.host_sdk_version`. Do not guess a version or use -`latest`. If already installed, inspect `inkcre-cli extension get inkcre/rss` before changing it. - -The type list should contain `extensions.rss.rss.Source` and `extensions.rss.atom.Source`. The Web -app's Extensions switch controls its browser runtime; it does not enable this Core collector. - -### Add and run the source - -1. Create `feed.json` in your local working folder: - - ```json - { - "nickname": "My first feed", - "config": { - "feed_url": "https://YOUR-PUBLICATION/FEED", - "fetch_full_text": false, - "download_enclosures": false - } - } - ``` - - Replace the URL. These first-run settings collect feed content without extra article fetches or - attachment downloads. - -2. Create the Source, choosing the matching feed format: - - ```sh - inkcre-cli source create --type extensions.rss.rss.Source --input feed.json - ``` - - For Atom, use `extensions.rss.atom.Source`. Record the returned Source `id`. - -3. Collect it. Replace `42` with your Source ID: - - ```sh - inkcre-cli source collect 42 --input-json '{}' - ``` - -4. The response contains a **Job ID**, different from the Source ID. Replace `17` with it: - - ```sh - inkcre-cli job wait 17 --for 30s - ``` - - Look for `status: finished`. If still `pending` or `running`, observe again rather than creating - another collect job. If `failed`, inspect `inkcre-cli job get 17 --json` and its diagnostics. - Ending observation does not stop the Job. - -**Checkpoint:** the collection Job finished. A feed exposes only what its publisher currently -provides; success does not mean its entire historical archive was imported. - -## 4. Find and inspect what you saved - -Collection and indexing are separate. Create a lexical-index maintenance Job after collection to -enable search by remembered words, without an AI provider: - -```sh -inkcre-cli job create --type core.feature_retrieval.lexical.maintain.v1 --input-json '{"parameters":{}}' -``` - -Use its returned Job ID with `inkcre-cli job wait JOB_ID --for 30s`, replacing `JOB_ID` with the -number. Check its status and `state` for indexed records and diagnostics. Maintenance processes a -bounded batch; repeat if you imported more than one batch can index. - -Copy a distinctive phrase from an item in your feed and search: - -```sh -inkcre-cli recall 'A phrase from your feed item' --mode lexical -``` - -Read a returned Block's content and relationships, replacing `123` with that Block's ID: - -```sh -inkcre-cli resolver invoke block:123 --method get_text -inkcre-cli graph neighborhood block:123 -``` - -**Checkpoint:** you can retrieve a real item you recognize and see its source relationships. You -have completed the first loop: source → saved information → information you can use. - -This first form of organization comes from the source: feed items belong to feeds, GitHub -repositories can belong to Lists, and mail has sender and mailbox relationships. It does not mean -InKCre has already summarized, tagged, or reorganized everything with AI. - -## 5. Keep collecting and searching - -After the manual run works, create `collect-hourly.json`, replacing `42` with your Source ID: - -```json -{ - "schedule": "0 * * * *", - "job_parameters": { "source": 42, "config": {} } -} -``` - -```sh -inkcre-cli cron create --job-type core.source.collect.v1 --input collect-hourly.json -``` - -This five-field schedule means “at minute zero of every hour.” Add independent index maintenance so -later items become searchable. Save this as `index-periodically.json`: - -```json -{ - "schedule": "*/10 * * * *", - "job_parameters": {} -} -``` - -```sh -inkcre-cli cron create --job-type core.feature_retrieval.lexical.maintain.v1 --input index-periodically.json -inkcre-cli cron list -``` - -Create each schedule once. Inspect its returned Cron ID with `inkcre-cli cron get ID`; `last_job` -identifies the Job to check. Pause it with `inkcre-cli cron disable ID`. Replace `ID` with the -actual number. Your terminal may be closed, but Core must run. Sleeping hosts miss occurrences and -do not automatically catch up. Independent indexing also means new items may not become searchable -immediately. - -## 6. Add the sources you actually use - -For each new source: enable its Core Extension, create the Source, collect once, inspect the Job, -run index maintenance, and search for a known item. Only then add a schedule. - -The versions below are published releases for Core Host `0.2.x`. Check the linked release listings -and your running type's `--schema` when versions differ. Source files and command output may contain -account credentials; do not publish them. - -### GitHub Stars and Lists - -1. Create a personal access token for the account whose Stars and Lists you want to collect, - following - [GitHub's token instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens). - Its account and permissions determine which data is visible. -2. Enable the Extension: - - ```sh - inkcre-cli extension install inkcre/github --version 0.3.0 - inkcre-cli extension enable inkcre/github - ``` - -3. Save `github.json` with your token: - - ```json - { - "nickname": "My GitHub saves", - "config": { "github_token": "YOUR-GITHUB-TOKEN" } - } - ``` - - ```sh - inkcre-cli source create --type extensions.github.stars.Source --input github.json - ``` - -4. Collect the returned Source ID as in step 3. After indexing, search for a repository you starred. - Each run synchronizes current Stars and Lists; it is not a code backup or a full GitHub activity - archive. - -See the [GitHub collector](https://github.com/InKCre/core-py/blob/main/extensions/github/README.md) -and [published releases](https://registry.inkcre.dev/v1/extensions/inkcre/github). - -### Email over IMAP - -1. Find your provider's IMAP hostname and enable IMAP if required. Obtain an app-specific password - where supported. Accounts requiring an unsupported authentication method cannot be connected by - substituting an ordinary login password. -2. Install and enable `inkcre/mail` version `0.3.0`, using the two Extension commands above with - `inkcre/mail` in place of `inkcre/github`. -3. Save `mail.json`, replacing the host and credentials: - - ```json - { - "nickname": "My mail", - "config": { - "protocol": "imap", - "parameters": { - "host": "YOUR-IMAP-HOST", - "port": 993, - "security": "tls", - "username": "YOUR-MAIL-LOGIN", - "password": "YOUR-APP-PASSWORD" - }, - "ordinary_mark_as_seen": false, - "synchronize_deletions": false - } - } - ``` - - ```sh - inkcre-cli source create --type extensions.mail.source.Source --input mail.json - ``` - -4. Send yourself a test email **after creating the Source**, then collect its ID. Ordinary - collection starts with new mail; an empty first run does not imply login failed. Inspect mailbox - diagnostics even if the Job finishes. This example preserves unread state; the collector's - default would mark ordinary collected mail as seen. -5. For older mail, request a small explicit backfill. Replace `42` with the Mail Source ID and - choose dates for your mailbox: - - ```sh - inkcre-cli source backfill 42 --input-json '{"since":"2026-09-01","before":"2026-09-08"}' - ``` - - The start date is included and the end date excluded. Observe the Job and index afterward. This - collector does not send mail or create email digests. - -See the -[Mail configuration](https://github.com/InKCre/core-py/blob/main/docs/30-unit-tdd/mail-extension.md) -and [published releases](https://registry.inkcre.dev/v1/extensions/inkcre/mail). - -### Send or forward messages from Telegram - -1. Create a dedicated bot with [@BotFather](https://t.me/botfather): send `/newbot`, follow its - naming prompts, and retain the token. Obtain your own numeric user ID using - [Telegram Desktop's data export](https://telegram.org/blog/export-and-more): open **Settings → - Advanced → Export Telegram data**, include personal information, and choose JSON format. In the - exported JSON, use `personal_information.user_id`, not your `@username` or the bot's ID. Telegram - may require confirmation from another device or a waiting period. -2. Install and enable `inkcre/telegram` version `0.3.0` with the same Extension commands. -3. Save `telegram.json`, replacing the token and example user ID: - - ```json - { - "nickname": "My Telegram inbox", - "config": { - "bot_token": "YOUR-BOT-TOKEN", - "bound_user_id": 123456789, - "download_attachments": false - } - } - ``` - - ```sh - inkcre-cli source create --type extensions.telegram.source.Source --input telegram.json - ``` - -4. Send a distinctive text message in a private chat with the bot, then collect the Source ID. - Successfully saved messages receive a 👍 reaction. After indexing, search for its text. -5. Add a schedule to forward messages without running the CLI each time. `*/5 * * * *` polls every - five minutes while Core is awake. Telegram retains updates for a limited time, so a long-sleeping - instance should not be your only copy. - -Use one bot for one Source. This is a capture inbox, not group/channel history import or a -notification destination. Attachments are metadata-only in this example; enable -`download_attachments` when you want their bytes saved too. See the -[Telegram collector](https://github.com/InKCre/core-py/blob/main/extensions/telegram/README.md) and -[published releases](https://registry.inkcre.dev/v1/extensions/inkcre/telegram). - -Add more RSS/Atom subscriptions with one Source per feed. Memos-compatible capture is also available -through a separately configured -[Memos Extension](https://github.com/InKCre/core-py/blob/main/docs/30-unit-tdd/memos-extension.md). -An arbitrary service or browser account is not automatically a supported Source; check for a -collector before assuming its history can be imported. - -## 7. Bring information into your daily workflow - -### Read and explore in the Web app - -The browser needs its **own** Client ID. Do not reuse the Core Peer ID: the browser excludes its own -identity when looking for another Peer to execute a capability. Settings does not yet create this -record for you. - -1. In your dedicated Neon project's **SQL Editor**, select the same default branch and `neondb` - database as deployment. For a manually hosted database, use your PostgreSQL SQL client against - the initialized InKCre database instead. Run this once to register your browser: - - ```sql - INSERT INTO inkcre.peers (id, name, config) - VALUES ( - gen_random_uuid(), - 'My web browser', - '{"extension_registry_url":"https://registry.inkcre.dev"}'::jsonb - ) - RETURNING id; - ``` - - Save the returned UUID as your browser Client ID. This adds one record without replacing Core's - record. Reuse it when reconnecting this browser. - -2. Wake Core `/readyz`, then open [Web app Settings](https://app.inkcre.dev/settings). -3. Enter **PostgreSQL REST URL** = PostgREST base URL, **JWT Secret** = your saved secret, and - **Client ID** = the new browser UUID. Choose **Save**, then reload. The Clients list should - include Core, reporting online. -4. Open **Info Base**, search for the phrase that worked in step 4, and inspect a result. Use **View - content** for supported content and the graph view to follow relationships. The list is a search - surface, not every stored Block. Some source renderers need a compatible Web Extension; the CLI's - `get_text` remains a way to read Core-resolved text. -5. Bookmark the app. Another browser/device needs its own connection setup. **Export** omits the - secret and is not an info-base backup. - -### Use your information from a terminal or AI tool - -Keep `inkcre-cli recall 'your clue' --mode lexical` available wherever you work. A terminal-based -assistant you trust can use the installed CLI and named connection. For example: - -> Use my InKCre personal connection to find articles I saved about distributed systems. Read the -> matches and cite their original links. Do not change my sources or data. - -The connection has owner authority; only give it to a tool you trust. Its model provider may receive -the content it reads. - -For clients that connect over MCP instead of running terminal commands, follow the -[MCP Sink setup](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/mcp-sink.md). Create -and enable a `core.mcp.v1` Sink with a separate PAT, then configure Bearer authentication for -`https://YOUR-CORE-HOST/sinks/SINK_ID/mcp`. This needs explicit Sink and client setup; the CLI has -no `sink` command. The MCP PAT is not `JWT_SECRET`. Check the client's authentication support or the -documented tunnel before choosing this route. - -These paths make your saved information available on demand. They do not configure proactive -notifications or a daily briefing. - -### Add AI organization when you need it - -Once collection and reading work, configure a model provider and Agent for rumination, then -explicitly reconsider a Block. Follow the -[Core organization configuration](https://github.com/InKCre/core-py/blob/main/docs/30-unit-tdd/organization.md) -for the Agent and deployment settings before using **Ruminate** or -`inkcre-cli organization ruminate BLOCK_ID`. - -Inspect the Job and graph afterward. A valid result may add nothing. Re-index after new content is -added to find it by words. Semantic search additionally needs an embedding provider/profile and -maintenance; an LLM API key alone does not enable it. Selected content leaves your instance for -configured AI providers and may incur charges. - -## If a step does not work - -| What you observe | What to check next | -| ---------------------------------------- | --------------------------------------------------------------------------------------------------------- | -| Deployment workflow missing | Enable Actions in your fork and check that it is up to date with upstream `main`. | -| Core slow or offline | Open Core `/readyz`; inspect host logs if it stays unready. PostgREST may sleep separately. | -| Client returns `401` | Check endpoint and secret. Anonymous PostgREST `401` is expected; authenticated `401` is not. | -| Source type absent | Install a compatible Extension and enable it on Core, not only in the Web app. | -| Job stays pending | Confirm Core is awake and its collector enabled. Job creation only confirms acceptance. | -| Collection finished but search empty | Confirm the source exposed the item; run lexical maintenance and inspect diagnostics. | -| CLI works but Web cannot find a provider | Check Core readiness and that browser and Core Peer IDs differ. | -| Old mail missing | Ordinary collection starts with new mail. Request a historical backfill. | -| Truncated content or no attachment bytes | First-run examples avoid extra downloads. Check the collector's settings. | -| Request outcome uncertain | Read the Job or resulting data before repeating a write; a lost response does not prove nothing happened. | - -Keep hosting and connection credentials in your password manager. Review your database's -backup/restore options and hosting usage before collecting irreplaceable information. A fork, a -client configuration export, and a source account are not backups of your info-base. +If a step fails, use [Troubleshooting](/guide/troubleshooting). For backups, upgrades, and ongoing +hosting responsibilities, return to [Advanced Self-Hosting](/self-hosted/advanced). diff --git a/website/content/en/self-hosted/heroku-neon.md b/website/content/en/self-hosted/heroku-neon.md new file mode 100644 index 0000000..a3a8a72 --- /dev/null +++ b/website/content/en/self-hosted/heroku-neon.md @@ -0,0 +1,56 @@ +--- +title: Deploy with Heroku and Neon +description: Deploy your own InKCre instance using a GitHub fork, Neon, and Heroku. +--- + +# Deploy with Heroku and Neon + +Use this fork-based quick deployment if you do not have an instance yet. You need GitHub, Neon, and +a compute-provider account. These providers are convenience options, not InKCre requirements; +[Advanced Self-Hosting](/self-hosted/advanced) covers your own infrastructure. + +Keep credentials in a password manager. API keys let the workflow act on your hosting accounts; they +are different from sign-in passwords. + +Choose Heroku instead of Render; you do not need both providers. The database still lives in Neon, +while Heroku runs two apps: Core and PostgREST. + +1. Open [InKCre/core-py](https://github.com/InKCre/core-py), choose **Fork**, and enable workflows + in your fork's **Actions** tab. Create a dedicated project in [Neon](https://console.neon.tech), + keeping the default `neondb` database and `neondb_owner` role. Save its project ID and create a + Neon API key with access to it. +2. Create a [Heroku account](https://dashboard.heroku.com/) with billing enabled and obtain an API + key. The workflow creates the two apps; you do not need to provision a Heroku database add-on. +3. In your fork's **Settings → Secrets and variables → Actions**, add the following settings. + Generate independent random values for the three credential secrets and retain them in your + password manager. + +| Kind | Name | Value | +| -------- | ----------------------------- | ---------------------------------------------------------------------------------- | +| Secret | `NEON_API_KEY` | Your Neon API key. | +| Secret | `HEROKU_API_KEY` | Your Heroku API key. | +| Secret | `JWT_SECRET` | A new random secret of at least 32 ASCII characters. | +| Secret | `CORE_DATABASE_PASSWORD` | A separate random password of at least 32 ASCII characters. | +| Secret | `POSTGREST_DATABASE_PASSWORD` | Another random password of at least 32 ASCII characters. | +| Variable | `NEON_PROJECT_ID` | Your Neon project ID. | +| Variable | `HEROKU_APP_PREFIX` | A unique lowercase app prefix, such as `alex-inkcre`, between 3 and 18 characters. | + +4. Open **Actions → Deploy self-hosted InKCre to Heroku → Run workflow**, select your fork's `main` + branch, and wait for the run to succeed. +5. Save the Core URL, PostgREST URL, and Core Peer ID from its summary. Open Core `/readyz` and wait + for HTTP `200`, then continue to [Connect the CLI](/guide/connect-cli). + +The workflow runs one Eco web dyno per app. Heroku charges and +[Eco sleep behavior](https://devcenter.heroku.com/articles/eco-dyno-hours) apply. Keep the same +database passwords on reruns; replacing them is a coordinated credential rotation, not a routine +redeploy. As with Render Free, sleeping Core cannot provide continuous collection. The maintained +deployment procedure and recovery details live in the +[Core Heroku self-hosting guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/heroku-neon-self-host.md). + +**Keep your instance credentials private.** `JWT_SECRET` grants control of the instance, not a +limited personal login. Do not paste it into an online JWT generator, an issue, a chat, or your +public fork. This deployment does not isolate different users from one another. Start with your own +information and trusted devices. + +Next: [Connect the CLI](/guide/connect-cli), or return to the +[self-hosted walkthrough](/self-hosted/getting-started). diff --git a/website/content/en/self-hosted/index.md b/website/content/en/self-hosted/index.md index 094df73..a53bfc0 100644 --- a/website/content/en/self-hosted/index.md +++ b/website/content/en/self-hosted/index.md @@ -16,11 +16,11 @@ For the application-level introduction and access choices, start with ## Choose a deployment path -| Path | What you manage | Start here | -| ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | -| Fork quick deployment: Neon + Render | Provider accounts and GitHub settings; the workflow initializes the database and deploys Core and PostgREST. | [Render walkthrough](/self-hosted/getting-started#render-neon) | -| Fork quick deployment: Neon + Heroku | The same topology on Heroku, with its billing and runtime settings. | [Heroku walkthrough](/self-hosted/getting-started#heroku-neon) | -| Manual deployment | PostgreSQL initialization, PostgREST, the Core Python process or container, and hosting operations on infrastructure you choose. | [Advanced](/self-hosted/advanced#manual-deployment) | +| Path | What you manage | Start here | +| ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------- | +| Fork quick deployment: Neon + Render | Provider accounts and GitHub settings; the workflow initializes the database and deploys Core and PostgREST. | [Render walkthrough](/self-hosted/render-neon) | +| Fork quick deployment: Neon + Heroku | The same topology on Heroku, with its billing and runtime settings. | [Heroku walkthrough](/self-hosted/heroku-neon) | +| Manual deployment | PostgreSQL initialization, PostgREST, the Core Python process or container, and hosting operations on infrastructure you choose. | [Advanced](/self-hosted/advanced) | Neon, Render, and Heroku are convenient deployment options, not product dependencies. Forking provides a ready-made deployment workflow; it is not required to run InKCre. Both quick deployments @@ -33,5 +33,4 @@ Follow [Getting Started](/self-hosted/getting-started) to deploy an instance, co find a saved item, and add the sources you use. The guide then connects the instance to the Web app and everyday tools. It assumes basic technical familiarity but no InKCre development setup. -If you already operate a compatible instance, begin at -[connecting the CLI](/self-hosted/getting-started#connect-cli). +If you already operate a compatible instance, begin at [connecting the CLI](/guide/connect-cli). diff --git a/website/content/en/self-hosted/render-neon.md b/website/content/en/self-hosted/render-neon.md new file mode 100644 index 0000000..b3d1edc --- /dev/null +++ b/website/content/en/self-hosted/render-neon.md @@ -0,0 +1,58 @@ +--- +title: Deploy with Render and Neon +description: Deploy your own InKCre instance using a GitHub fork, Neon, and Render. +--- + +# Deploy with Render and Neon + +Use this fork-based quick deployment if you do not have an instance yet. You need GitHub, Neon, and +a compute-provider account. These providers are convenience options, not InKCre requirements; +[Advanced Self-Hosting](/self-hosted/advanced) covers your own infrastructure. + +Keep credentials in a password manager. API keys let the workflow act on your hosting accounts; they +are different from sign-in passwords. + +1. Open [InKCre/core-py](https://github.com/InKCre/core-py), choose **Fork**, and create your + repository copy. Open its **Actions** tab and enable workflows if GitHub asks. +2. Create a project in [Neon](https://console.neon.tech). Keep the default `neondb` database and + `neondb_owner` role. Save the project ID and create a Neon API key with access to it. +3. Create a workspace in [Render](https://dashboard.render.com). Save its workspace ID from Settings + and create a Render API key. Your fork must be public, or Render must already have permission to + read it. +4. In your fork, open **Settings → Secrets and variables → Actions**. Use **New repository secret** + under **Secrets** and **New repository variable** under **Variables** to add: + +| Kind | Name | Value | +| -------- | ----------------------- | ------------------------------------------------------------------------------------ | +| Secret | `NEON_API_KEY` | Your Neon API key. | +| Secret | `RENDER_API_KEY` | Your Render API key. | +| Secret | `JWT_SECRET` | A new random secret of at least 32 ASCII characters, saved in your password manager. | +| Variable | `NEON_PROJECT_ID` | Your Neon project ID. | +| Variable | `RENDER_OWNER_ID` | Your Render workspace ID. | +| Variable | `RENDER_SERVICE_PREFIX` | A unique lowercase prefix such as `alex-inkcre`, between 3 and 40 characters. | + +5. Open **Actions → Deploy self-hosted InKCre → Run workflow**, select your fork's `main` branch, + and run it. Wait for completion. If it fails, open the failed step, correct the reported problem, + and rerun using the same credentials. +6. Open the completed run's summary. Save the **Core URL**, **PostgREST URL**, and **Core Peer ID**. + The workflow deliberately does not print your secret; retain the original value. +7. Open the Core URL with `/readyz` appended. Continue when it returns HTTP `200`. A sleeping + service may take time to start. + +**Checkpoint:** deployment succeeded, and you have both service URLs and your `JWT_SECRET`. Use the +**Core URL** for the CLI and the **PostgREST URL** for the Web app later. + +The workflow selects Render Free services. They sleep when idle, share account usage limits, and +cannot guarantee continuous collection. Check +[Render's current limits](https://render.com/docs/free) and your Neon plan before relying on them. +Continuous scheduled collection needs hosting that keeps Core running. The maintained deployment +procedure and recovery details live in the +[Core self-hosting guide](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/render-neon-self-host.md). + +**Keep your instance credentials private.** `JWT_SECRET` grants control of the instance, not a +limited personal login. Do not paste it into an online JWT generator, an issue, a chat, or your +public fork. This deployment does not isolate different users from one another. Start with your own +information and trusted devices. + +Next: [Connect the CLI](/guide/connect-cli), or return to the +[self-hosted walkthrough](/self-hosted/getting-started). From 64784736552866ad90616cc457a67535f3b2e416 Mon Sep 17 00:00:00 2001 From: Lan_zhijiang Date: Sun, 20 Sep 2026 16:19:46 +0800 Subject: [PATCH 06/18] =?UTF-8?q?docs(sinks):=20=E5=A2=9E=E5=8A=A0=20ChatG?= =?UTF-8?q?PT=20MCP=20=E4=B8=8E=20tunnel=20=E9=85=8D=E7=BD=AE=E6=95=99?= =?UTF-8?q?=E7=A8=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 覆盖 Sink 创建、凭据分离、tunnel 鉴权和 ChatGPT 连接 - 以真实工具检索作为验收点，说明 Skill 导入限制 - 增加停用步骤与常见故障排查 --- website/.vitepress/config.mts | 6 +- website/content/en/guide/daily-use.md | 16 +- website/content/en/guide/sinks/chatgpt.md | 196 ++++++++++++++++++++++ 3 files changed, 211 insertions(+), 7 deletions(-) create mode 100644 website/content/en/guide/sinks/chatgpt.md diff --git a/website/.vitepress/config.mts b/website/.vitepress/config.mts index 545c138..80e9359 100644 --- a/website/.vitepress/config.mts +++ b/website/.vitepress/config.mts @@ -65,7 +65,11 @@ export default defineConfig({ { text: 'Find What You Saved', link: '/guide/search' }, { text: 'Schedule Collection and Indexing', link: '/guide/schedules' }, { text: 'Connect More Sources', link: '/guide/sources' }, - { text: 'Use Your Information', link: '/guide/daily-use' }, + { + text: 'Use Your Information', + link: '/guide/daily-use', + items: [{ text: 'Sinks: ChatGPT via MCP', link: '/guide/sinks/chatgpt' }], + }, { text: 'Troubleshooting', link: '/guide/troubleshooting' }, ], }, diff --git a/website/content/en/guide/daily-use.md b/website/content/en/guide/daily-use.md index 5b57b38..b1fc69b 100644 --- a/website/content/en/guide/daily-use.md +++ b/website/content/en/guide/daily-use.md @@ -54,12 +54,16 @@ assistant you trust can use the installed CLI and named connection. For example: The connection has owner authority; only give it to a tool you trust. Its model provider may receive the content it reads. -For clients that connect over MCP instead of running terminal commands, follow the -[MCP Sink setup](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/mcp-sink.md). Create -and enable a `core.mcp.v1` Sink with a separate PAT, then configure Bearer authentication for -`https://YOUR-CORE-HOST/sinks/SINK_ID/mcp`. This needs explicit Sink and client setup; the CLI has -no `sink` command. The MCP PAT is not `JWT_SECRET`. Check the client's authentication support or the -documented tunnel before choosing this route. +## Connect a Sink to ChatGPT + +A Sink exposes InKCre capabilities to an external tool. Follow +[Connect ChatGPT through MCP](/guide/sinks/chatgpt) to create the MCP Sink, run Secure MCP Tunnel, +add the ChatGPT connection, and verify a real retrieval. It explains the separate credentials and +what must keep running; no CLI `sink` command is required. + +For another MCP host that can send a Bearer PAT directly, the +[Core MCP Sink reference](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/mcp-sink.md) +owns the endpoint and authentication contract. These paths make your saved information available on demand. They do not configure proactive notifications or a daily briefing. diff --git a/website/content/en/guide/sinks/chatgpt.md b/website/content/en/guide/sinks/chatgpt.md new file mode 100644 index 0000000..902c1c3 --- /dev/null +++ b/website/content/en/guide/sinks/chatgpt.md @@ -0,0 +1,196 @@ +--- +title: Connect ChatGPT through MCP +description: Use the InKCre MCP Sink and Secure MCP Tunnel to retrieve your information in ChatGPT. +--- + +# Connect ChatGPT through MCP + +A **Sink** makes InKCre capabilities available to another tool. This guide connects the MCP Sink to +ChatGPT through OpenAI's Secure MCP Tunnel, so you can ask ChatGPT to find and read information from +your instance. It is an on-demand connection, not a scheduled digest or notification service. + +The connection follows this path: + +```text +ChatGPT → OpenAI Secure MCP Tunnel → your running tunnel-client → InKCre MCP Sink +``` + +The tunnel client must be able to reach Core. It can run on your computer or another machine you +operate; it does not have to run beside Core. Keep it running whenever ChatGPT needs the connection. + +## Before you start + +- Have a working instance and complete [a known-item search](/guide/search) first. Retain its Core + URL and private JWT secret. If someone else operates the instance, ask them to perform the Sink + setup rather than requesting their administrator credentials. +- Install the CLI environment from [Connect the CLI](/guide/connect-cli). The setup example below + uses its installed Python dependencies because the CLI does not yet have a `sink` command. +- Confirm that your ChatGPT account/workspace permits developer-mode MCP connections and that you + can access [Platform Tunnels](https://platform.openai.com/settings/organization/tunnels). + Availability and UI labels depend on OpenAI's rollout and workspace policy; check the current + [developer-mode requirements](https://help.openai.com/en/articles/12584461-developer-mode-and-mcp-apps-in-chatgpt) + if an option is missing. + +Connecting authorizes ChatGPT to use the Sink's exposed capabilities and receive returned content. +Only connect an instance whose information you intend to share with that ChatGPT account/workspace. +The Sink PAT is not a per-user data-isolation boundary. Keep the tunnel machine and its credentials +under your control; someone who obtains the PAT can invoke the Sink if they can reach its endpoint. + +Keep these credentials distinct: + +| Value | Purpose | Where it belongs | +| ----------------------------- | --------------------------------------------------------- | --------------------------------------------------------------------------- | +| InKCre `JWT_SECRET` | Signs administrator requests to create and enable a Sink. | Local setup only; never enter it into ChatGPT or tunnel settings. | +| MCP Sink PAT | Admits calls to this MCP endpoint. | Sink configuration and the tunnel client's downstream Authorization header. | +| OpenAI tunnel runtime API key | Lets the client use the provisioned tunnel. | Tunnel client; not the Sink config. | + +## 1. Create and enable the MCP Sink + +Create a new random PAT in your password manager and save it. Do not reuse `JWT_SECRET` or an OpenAI +key. If you already have an enabled MCP Sink and its PAT, reuse them and skip this step. + +In a private local folder, save this as `create-mcp-sink.py`. Activate the CLI's Python environment +and run `python create-mcp-sink.py`. The prompts hide both secrets; nothing is stored in the script. + +```python +import getpass +import time + +import httpx +import jwt + +core_url = input("Core HTTPS URL: ").strip().rstrip("/") +if not core_url.startswith("https://"): + raise ValueError("Use your trusted Core HTTPS URL, not the PostgREST URL") +secret = getpass.getpass("InKCre JWT_SECRET: ") +pat = getpass.getpass("New MCP Sink PAT from your password manager: ") +if not secret or not pat: + raise ValueError("Both credentials are required") +issued = int(time.time()) - 5 +token = jwt.encode( + {"role": "authenticated", "iss": "inkcre-peer", "aud": "inkcre-api", + "iat": issued, "exp": issued + 900}, + secret, algorithm="HS256", +) +with httpx.Client(timeout=30, headers={"Authorization": f"Bearer {token}"}) as client: + created = client.post(f"{core_url}/sinks", json={ + "type": "core.mcp.v1", "nickname": "ChatGPT", "config": {"pat": pat}, + }) + created.raise_for_status() + sink_id = created.json()["id"] + print(f"Created Sink ID: {sink_id}; retain this ID before continuing.") + enabled = client.post(f"{core_url}/sinks/{sink_id}/enable") + enabled.raise_for_status() + print(f"MCP endpoint: {core_url}/sinks/{sink_id}/mcp") +``` + +Save the printed Sink ID and MCP endpoint. Enabling applies to the Core Peer you contacted. Do not +append this path to the PostgREST URL or to `/readyz`. + +If a request times out or enabling fails, inspect the existing Sink before repeating creation: the +first write may already have succeeded. The operator can use authenticated `GET /sinks` and +`POST /sinks/{id}/enable` with the same short-lived Bearer JWT pattern. Do not publish a Sink +management response; it can contain configuration credentials. The authoritative API and lifecycle +details are in +[MCP Sink Operations](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/mcp-sink.md). + +## 2. Provision a tunnel and install its client + +1. Open [Platform Tunnels](https://platform.openai.com/settings/organization/tunnels) in the + organization associated with your ChatGPT account/workspace. Create a tunnel and retain its + `tunnel_id`. If tunnel creation is unavailable, ask the organization administrator for access. +2. Obtain a runtime API key whose principal can **Read** and **Use** that tunnel. Creating tunnels + needs **Manage** permission; do not use an admin API key for the long-running client. +3. Download the supported `tunnel-client` for your operating system from the Tunnels page, extract + it, and place the executable on your `PATH`. Verify `tunnel-client --version` in a terminal. + +The +[official tunnel onboarding guide](https://github.com/openai/tunnel-client/blob/master/docs/onboarding.md) +owns account setup, downloads, and permissions. Keep the tunnel in the organization/workspace that +your ChatGPT session can access; an otherwise healthy tunnel in another account will not appear. + +## 3. Start the tunnel client + +The following commands use **Bash** on macOS/Linux or in WSL on Windows. Run `bash` first if your +terminal uses another shell. Enter secrets at the hidden prompts instead of pasting them into +commands saved in shell history: + +```bash +read -r -s -p 'MCP Sink PAT: ' INKCRE_MCP_PAT; printf '\n' +export INKCRE_MCP_AUTHORIZATION="Bearer $INKCRE_MCP_PAT" +unset INKCRE_MCP_PAT +read -r -s -p 'OpenAI tunnel runtime API key: ' CONTROL_PLANE_API_KEY; printf '\n' +export CONTROL_PLANE_API_KEY + +tunnel-client run \ + --control-plane.tunnel-id='YOUR-TUNNEL-ID' \ + --control-plane.api-key='env:CONTROL_PLANE_API_KEY' \ + --mcp.server-url='https://YOUR-CORE-HOST/sinks/YOUR-SINK-ID/mcp' \ + --mcp.extra-headers='Authorization: env:INKCRE_MCP_AUTHORIZATION' \ + --mcp.discovery-extra-headers='Authorization: env:INKCRE_MCP_AUTHORIZATION' \ + --health.listen-addr='127.0.0.1:8080' +``` + +Replace the tunnel ID and server URL before running. The environment value contains the complete +`Bearer ` header value: the `env:` reference does not add the Bearer prefix. Both header flags +are needed because discovery/probes and regular calls reach the protected Sink separately. See the +[tunnel configuration reference](https://github.com/openai/tunnel-client/blob/master/docs/configuration.md). + +In another terminal, run `curl --fail http://127.0.0.1:8080/readyz`; continue when it returns +HTTP 200. The local diagnostics UI is at `http://127.0.0.1:8080/ui`. If port 8080 is occupied, +choose another loopback port in the command and both URLs. Keep this listener local; do not expose +its UI as the MCP endpoint. Core must remain reachable too, including when hosted on a sleeping +plan. + +## 4. Add the connection in ChatGPT + +1. In ChatGPT, enable **Developer mode** in **Settings → Security and login**. Some workspace + interfaces expose it through Apps settings instead; follow the linked developer-mode help for + your account if the labels differ. +2. Open [ChatGPT Plugins](https://chatgpt.com/plugins), select the add button, and name the + connection `InKCre`. Under **Connection**, choose **Tunnel** and select your tunnel or enter its + ID. +3. Create the connection and review its discovered tools. If your interface offers **Scan Tools**, + run it. Expect InKCre tools such as `inkcre_recall` and `inkcre_read_blocks`, not the tunnel + client's embedded demo tools. Do not paste the InKCre JWT secret into ChatGPT. +4. Start a new chat using **Try in chat**, or select InKCre in the message's tools menu. + +These steps follow OpenAI's +[connection guide](https://developers.openai.com/plugins/deploy/connect-chatgpt). This is a private +developer-mode connection, not publication in a public plugin directory. + +## 5. Verify a real retrieval + +Choose an item that you already found with the CLI and ask: + +> Use InKCre to search for “[a distinctive phrase from my saved item]” using lexical retrieval. Read +> the most relevant result, summarize it, and cite its original source. Do not change my data. + +Confirm that ChatGPT actually calls InKCre tools, reads the expected item, and grounds its answer in +that content. A plausible answer without a tool call is not a connection test. Select InKCre again +for a later message that needs new retrieval; do not assume selection persists across messages. + +The previous InKCre acceptance exercised real ChatGPT tool calls through this tunnel path. It did +**not** establish automatic Skill import or proactive use: the server's `use-inkcre` Skill was +readable by an MCP client, but the tested ChatGPT scan did not import it. Treat discovered tools and +a successful real task as the checkpoint, not the presence of a Skill. Semantic retrieval separately +requires an embedding configuration; an unavailable semantic mode does not invalidate lexical +results. + +## Troubleshooting and stopping access + +| Symptom | What to check | +| ------------------------------------------ | ----------------------------------------------------------------------------------------- | +| Tunnel is not selectable | Match the Platform organization and ChatGPT account/workspace; verify tunnel permissions. | +| Sink responds `401` | Check the PAT and complete Bearer prefix in both runtime and discovery headers. | +| Sink responds `404` | Check Sink ID, Core origin, and whether it is enabled on that Peer. | +| Local tunnel readiness fails | Inspect its local UI/logs, runtime-key permissions, and reachability of Core. | +| Tools appear but ChatGPT does not use them | Attach InKCre to that message and request a known-item retrieval explicitly. | +| Search finds nothing | Repeat the CLI search and index maintenance from [Find What You Saved](/guide/search). | + +Refresh the ChatGPT connection after changing server tool metadata, then test in a new chat. +Stopping `tunnel-client` interrupts this tunnel path; closing its terminal or sleeping its host does +the same. Remove the connection from ChatGPT when no longer needed. To revoke the endpoint itself, +the operator can call authenticated `POST /sinks/{id}/disable`; disable before deleting a Sink. For +a leaked PAT, rotate the Sink config and update the tunnel environment before restarting it. Never +disable PAT authentication just to make discovery pass. From 2163c3a44c3e0299f4e1a338beeaded5d564951d Mon Sep 17 00:00:00 2001 From: Lan_zhijiang Date: Sun, 20 Sep 2026 16:52:45 +0800 Subject: [PATCH 07/18] =?UTF-8?q?docs(onboarding):=20=E6=8B=86=E5=88=86?= =?UTF-8?q?=E6=9D=A5=E6=BA=90=E6=95=99=E7=A8=8B=E5=B9=B6=E8=A1=A5=E5=85=85?= =?UTF-8?q?=E7=94=9F=E6=80=81=E6=89=A9=E5=B1=95=E5=BC=80=E5=8F=91=E6=8C=87?= =?UTF-8?q?=E5=8D=97?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 为 RSS、GitHub、邮件、Telegram、Twitter 和 Memos 提供独立上手路径 - 分离首次采集入口与共享 Job 操作，增加 Source Extension 构建及私有预览教程 - 核对发行版本并验证示例打包、授权请求和网站构建 --- website/.vitepress/config.mts | 21 +- website/README.md | 5 +- website/content/en/developer/architecture.md | 6 +- website/content/en/developer/contributing.md | 4 + .../content/en/developer/ecosystem/index.md | 45 +++ .../developer/ecosystem/source-extension.md | 286 ++++++++++++++++++ website/content/en/developer/index.md | 12 +- website/content/en/getting-started.md | 8 +- website/content/en/guide/collect.md | 36 +++ website/content/en/guide/first-source.md | 86 ++---- website/content/en/guide/search.md | 4 +- website/content/en/guide/sources.md | 159 ++-------- website/content/en/guide/sources/github.md | 45 +++ website/content/en/guide/sources/mail.md | 66 ++++ website/content/en/guide/sources/memos.md | 67 ++++ website/content/en/guide/sources/rss.md | 63 ++++ website/content/en/guide/sources/telegram.md | 57 ++++ website/content/en/guide/sources/twitter.md | 159 ++++++++++ .../content/en/self-hosted/getting-started.md | 3 +- 19 files changed, 923 insertions(+), 209 deletions(-) create mode 100644 website/content/en/developer/ecosystem/index.md create mode 100644 website/content/en/developer/ecosystem/source-extension.md create mode 100644 website/content/en/guide/collect.md create mode 100644 website/content/en/guide/sources/github.md create mode 100644 website/content/en/guide/sources/mail.md create mode 100644 website/content/en/guide/sources/memos.md create mode 100644 website/content/en/guide/sources/rss.md create mode 100644 website/content/en/guide/sources/telegram.md create mode 100644 website/content/en/guide/sources/twitter.md diff --git a/website/.vitepress/config.mts b/website/.vitepress/config.mts index 80e9359..151da2d 100644 --- a/website/.vitepress/config.mts +++ b/website/.vitepress/config.mts @@ -64,7 +64,19 @@ export default defineConfig({ { text: 'Collect Your First Source', link: '/guide/first-source' }, { text: 'Find What You Saved', link: '/guide/search' }, { text: 'Schedule Collection and Indexing', link: '/guide/schedules' }, - { text: 'Connect More Sources', link: '/guide/sources' }, + { + text: 'Connect More Sources', + link: '/guide/sources', + items: [ + { text: 'RSS and Atom', link: '/guide/sources/rss' }, + { text: 'GitHub Stars and Lists', link: '/guide/sources/github' }, + { text: 'Email over IMAP', link: '/guide/sources/mail' }, + { text: 'Telegram Inbox', link: '/guide/sources/telegram' }, + { text: 'Twitter / X Bookmarks', link: '/guide/sources/twitter' }, + { text: 'Memos-Compatible Capture', link: '/guide/sources/memos' }, + { text: 'Run a Collection', link: '/guide/collect' }, + ], + }, { text: 'Use Your Information', link: '/guide/daily-use', @@ -85,6 +97,13 @@ export default defineConfig({ }, ], '/developer/': [ + { + text: 'Ecosystem Developers', + items: [ + { text: 'Build on InKCre', link: '/developer/ecosystem/' }, + { text: 'Build a Source Extension', link: '/developer/ecosystem/source-extension' }, + ], + }, { text: 'Developer Guide', items: [ diff --git a/website/README.md b/website/README.md index 36083e3..3048ed5 100644 --- a/website/README.md +++ b/website/README.md @@ -35,7 +35,10 @@ pnpm --dir website audit --audit-level high - `/guide/` leaf pages own reusable client, collection, retrieval, scheduling, source, daily-use, and troubleshooting procedures. Link to these pages from any onboarding path rather than to sections buried inside the self-hosted walkthrough. -- `/developer/` contains architecture and contribution guidance; `/about/` describes the project. +- `/guide/sources` selects independent source tutorials under `/guide/sources/`; `/guide/collect` + owns shared collection and Job observation. Memos is documented separately as write-in capture. +- `/developer/` separates ecosystem integration guidance under `/developer/ecosystem/` from + architecture and core contribution guidance; `/about/` describes the project. - Section indexes use trailing-slash routes, such as `/developer/`. - Leaf pages use lowercase ASCII kebab-case routes without an extension, such as `/developer/architecture`. diff --git a/website/content/en/developer/architecture.md b/website/content/en/developer/architecture.md index 41d9365..3e1ae4a 100644 --- a/website/content/en/developer/architecture.md +++ b/website/content/en/developer/architecture.md @@ -80,8 +80,10 @@ activity are separate states: - **enabled**: a particular client is permitted to run it; - **running**: the current runtime has started it and applied its side effects. -The lifecycle and ownership model exists, but a stable third-party package format and SDK guide are -not yet public contracts. +Core supports native Python wheels with versioned Registry releases and explicit Host compatibility. +The [Source Extension tutorial](/developer/ecosystem/source-extension) covers the current Core Host +0.2 path. Its Source programming interfaces still import Core modules; the delivery Toolkit is not a +standalone Source SDK or a promise of compatibility with every future Host. ### APIs diff --git a/website/content/en/developer/contributing.md b/website/content/en/developer/contributing.md index 038217b..6e44a0e 100644 --- a/website/content/en/developer/contributing.md +++ b/website/content/en/developer/contributing.md @@ -8,6 +8,10 @@ description: Route an InKCre change to its canonical owner and repository-specif InKCre welcomes code, documentation, design, and ecosystem contributions. Start by identifying the owner of the change; then follow that owner's current development and review contract. +If you want to connect another service without changing InKCre itself, use the +[ecosystem developer guide](/developer/ecosystem/) instead. Your Extension can live in your own +repository; it does not need to become a Core contribution. + > [!NOTE] There is no copied, universal setup or contribution procedure. Toolchains and commands > belong to the repository that runs and verifies them. diff --git a/website/content/en/developer/ecosystem/index.md b/website/content/en/developer/ecosystem/index.md new file mode 100644 index 0000000..cdd63a0 --- /dev/null +++ b/website/content/en/developer/ecosystem/index.md @@ -0,0 +1,45 @@ +--- +title: Ecosystem Developers +description: Add integrations to InKCre without contributing to its core implementation. +--- + +# Ecosystem Developers + +Build an integration for your own workflow or distribute it to other InKCre operators. You do not +need to contribute it to the Core repository. Start with the boundary your integration needs: + +| Goal | Integration path | +| -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | +| Collect from a service InKCre does not support yet | [Build a Source Extension](/developer/ecosystem/source-extension) | +| Use existing information from ChatGPT | [Connect the MCP Sink](/guide/sinks/chatgpt) | +| Build a new runtime around the shared info-base | Read the [Peer architecture](/developer/architecture); this requires the admitted database protocol, not arbitrary SQL access | +| Change InKCre's own runtime or product behavior | Follow [Contributing](/developer/contributing) | + +## What an Extension supplies + +A Source fetches and maps external information. A Resolver interprets stored content and relations. +A Sink makes information useful downstream. An Extension packages one or more such capabilities for +a particular Host; not every integration needs all three or a custom browser interface. + +The Python path uses a native wheel, the `inkcre.core.extensions` entry point, and an exact Registry +Release. Core installs that release, a Peer enables it, and the running Host activates its behavior. +These are separate steps. Browser code is a separate native distribution, not automatically produced +by a Python wheel. + +The Source tutorial targets **Core Host 0.2.x**. Its Python programming interfaces currently import +Core modules; they are not an independent, universally stable Source SDK. The Extension Toolkit +builds delivery metadata and preview registries; it does not run collectors or replace Core. + +## Trust and delivery + +An admitted Extension is trusted in-process code, not sandboxed user content. A malicious package +could access the runtime's information and credentials. Operators must review what they install; +Registry publication is not proof of isolation. Use a separate test deployment and non-sensitive +fixtures during development, with its own database and credentials. + +Package identity, Host compatibility, dependencies, and immutable releases are part of delivering a +usable integration. Keep your own package, tests, release history, and user setup guide in your +repository. A public Registry requires its operator's namespace and publishing authorization; a +private development preview does not grant those rights. + +Continue with [Build a Source Extension](/developer/ecosystem/source-extension). diff --git a/website/content/en/developer/ecosystem/source-extension.md b/website/content/en/developer/ecosystem/source-extension.md new file mode 100644 index 0000000..77d0a8b --- /dev/null +++ b/website/content/en/developer/ecosystem/source-extension.md @@ -0,0 +1,286 @@ +--- +title: Build a Source Extension +description: + Build an independent Python collector and try its wheel in a Core Host 0.2 test deployment. +--- + +# Build a Source Extension + +This is for Python developers adding an integration, not contributors changing Core. Your Extension +lives in its own repository. It runs inside a matching **Core Host 0.2.x** environment because +Source and graph APIs currently import Core's `app.*` modules. Installing the delivery Toolkit alone +does not provide those runtime APIs. + +We will collect one JSON document, store its text, and link it to a Source anchor. Repeating an +unchanged response does not add a new snapshot. This deliberately small example has no OAuth, +pagination, custom Resolver, or UI; it is not a general-purpose JSON importer. + +## 1. Prepare an isolated test setup + +Use Python 3.12 and a separate [Core deployment](/self-hosted/) with disposable data. Connect the +[CLI](/guide/connect-cli) to that instance and ensure it is running Host 0.2.x. Do not use your +personal information store to experiment with trusted in-process code. + +Prepare an HTTP endpoint reachable **from Core** that returns this JSON: + +```json +{ "id": "note-1", "text": "Notebook verification: blue heron" } +``` + +For a local-only experiment, save it as `note.json` in a new folder containing no other files and +serve that folder with `python -m http.server 8765 --bind 127.0.0.1`. Its URL is +`http://127.0.0.1:8765/note.json` only when Core runs on that same machine outside a container. A +remote Core needs an endpoint it can actually reach; use a non-sensitive HTTPS fixture under your +control instead of exposing private directories or assuming your laptop's localhost is remote Core. + +## 2. Create your own package + +Create this layout in a new project directory: + +```text +notebook-extension/ + pyproject.toml + extensions/ + notebook/ + __init__.py + source.py +``` + +Do **not** add `extensions/__init__.py`: `extensions` is a shared namespace package. Before +distributing, replace `yourname` with a namespace you control and choose a unique module/entry-point +name to avoid collisions with other installed Extensions. + +Save `pyproject.toml`: + +```toml +[build-system] +requires = ["setuptools>=80,<81"] +build-backend = "setuptools.build_meta" + +[project] +name = "yourname-inkcre-notebook" +version = "0.1.0" +description = "One-document notebook collector for InKCre" +requires-python = ">=3.12,<3.13" +dependencies = ["httpx>=0.28.1,<0.29", "pydantic>=2.10.6,<3"] + +[project.entry-points."inkcre.core.extensions"] +notebook = "extensions.notebook:Extension" + +[tool.inkcre-extension] +name = "yourname/notebook" +nickname = "Notebook" +host-sdk = "core-py" +host-sdk-version = ">=0.2.0 <0.3.0" + +[tool.setuptools.packages.find] +include = ["extensions.notebook*"] +namespaces = true +``` + +The product coordinate (`yourname/notebook`), Python project name, and module path serve different +purposes. The entry-point name and Extension's `ext_id` must agree. Declare direct dependencies; +Core checks them against its existing environment and will not fetch arbitrary missing dependencies +while enabling your Extension. A new dependency may require an operator-built Core image. + +Save `extensions/notebook/__init__.py`: + +```python +from app.business.extension.main import EmptyConfig, ExtensionBase + + +class Extension(ExtensionBase[EmptyConfig], ext_id="notebook", config_cls=EmptyConfig): + @classmethod + def _init_sources(cls): + from .source import Source # Registers the class when the Host starts it. +``` + +Import-time registration must not connect to the database or fetch provider data. The Host handles +startup and catalog synchronization. No extra HTTP endpoint or scheduler is needed. + +## 3. Implement collection + +Save `extensions/notebook/source.py`: + +```python +import httpx +from pydantic import BaseModel, ConfigDict, Field, HttpUrl + +from app.business.info_base.commands import persist_stars +from app.business.info_base.resolver import TextResolver +from app.business.source import SourceBase, SourceManager +from app.persistence.source.uow import source_uow +from app.schemas.info_base.relation import RelationModel +from app.schemas.job import JobModel + + +class SourceConfig(BaseModel): + model_config = ConfigDict(extra="forbid") + url: HttpUrl + + +class Note(BaseModel): + model_config = ConfigDict(extra="forbid", strict=True) + id: str = Field(min_length=1) + text: str = Field(min_length=1, max_length=100_000) + + +class Source(SourceBase[SourceConfig], config_cls=SourceConfig): + """Save changes to one operator-selected JSON document as text snapshots.""" + + async def collect(self, job: JobModel, config: BaseModel) -> None: + source_config = await self.get_config() + url = str(source_config.url) + async with httpx.AsyncClient(timeout=20) as client: + response = await client.get(url) + response.raise_for_status() + note = Note.model_validate(response.json()) + snapshot = {"url": url, **note.model_dump()} + + async with source_uow() as uow: + source = await uow.sources.get(self._id, lock=True) + if source is None: + raise ValueError("Source was deleted") + if SourceConfig.model_validate(source.config) != source_config: + raise ValueError("Source config changed during collection; run again") + if (source.state or {}).get("last_snapshot") == snapshot: + return + anchor = await SourceManager.ensure_block_async(source, uow) + block = await persist_stars(TextResolver.create_graph(note.text), uow.graph) + if anchor.id is None or block.id is None: + raise RuntimeError("Persisted Block has no ID") + await uow.graph.relations.fetchsert( + RelationModel(from_=anchor.id, to_=block.id, content="snapshot") + ) + # ponytail: remembers one snapshot; use native-ID reconciliation for a multi-item feed. + source.state = {**(source.state or {}), "last_snapshot": snapshot} + await uow.sources.save(source) +``` + +HTTP and input errors fail the Job instead of reporting a false success. Network I/O finishes before +the database transaction. The anchor, text, relation, and state commit together; state is not +advanced if persistence fails. URL participates in snapshot identity so changing the endpoint does +not reuse the old endpoint's state. The built-in text Resolver makes the saved text readable and +indexable. + +This is a snapshot collector: a changed response creates another snapshot; returning to older text +can create another one too. It neither reconciles a whole remote collection nor deletes earlier +snapshots. The endpoint is selected by the trusted operator, not exposed as a public URL-fetching +API. For a real service, add its authentication, bounded response handling, native identity, +pagination, rate limits, and incremental state according to that service's contract. + +Source configuration is long-lived input, Source state remembers progress, and Job parameters/state +belong to one execution. Use `collect_config_cls` for typed per-run options and +`backfill_config_cls` plus `backfill()` only when you actually implement a historical collection +mode. Keep indexing and organization separate from collection. + +## 4. Build and finalize the wheel + +In the project directory, create a build environment. The Toolkit is a delivery tool, not a runtime +Source SDK: + +```sh +python3.12 -m venv .venv +. .venv/bin/activate +python -m pip install build 'inkcre-extension-toolkit[cli] @ https://github.com/InKCre/ext-reg/releases/download/toolkit-v0.2.1/inkcre_extension_toolkit-0.2.1-py3-none-any.whl' +python -m build --wheel --outdir dist/raw +inkcre-ext python wheel finalize --project pyproject.toml --wheel dist/raw/yourname_inkcre_notebook-0.1.0-py3-none-any.whl --output-dir dist/final +``` + +On Windows, activate with `.venv\Scripts\Activate.ps1`. If you renamed the project, substitute the +actual wheel filename. Finalization adds the installed `.dist-info/inkcre-extension.json` metadata +required by Core. Keep raw and finalized output separate; ship the finalized wheel. + +## 5. Try a private preview without publisher credentials + +Save `preview.json` beside `pyproject.toml`: + +```json +{ + "schema_version": 1, + "distributions": [ + { + "kind": "python", + "producer": "pyproject.toml", + "artifact": "dist/final/yourname_inkcre_notebook-0.1.0-py3-none-any.whl" + } + ] +} +``` + +For Core running on the same machine, build and serve the preview in a separate terminal: + +```sh +inkcre-ext preview build --inventory preview.json --public-origin http://127.0.0.1:8766 --output dist/registry +python -m http.server 8766 --bind 127.0.0.1 --directory dist/registry +``` + +For remote Core, publish **only** the generated `dist/registry` directory to an isolated static +HTTPS origin you control and use that origin as `--public-origin`. Keep it available for +installation and restart. This facade contains only the supplied releases; it is not a mirror of the +public Registry. + +In the terminal with your CLI, inspect `inkcre-cli peer get self` and +`inkcre-cli config get extension.registry`. Record the prior setting (a missing config is normal). A +Peer-level `extension_registry_url` override takes precedence; use a test Peer without an override +or have its operator adjust that override. On this **isolated test instance only**, set the +deployment Registry origin, replacing the URL if Core is remote: + +```sh +inkcre-cli config replace extension.registry --schema-id extension.registry.config.v1 --input-json '{"extension_registry_url":"http://127.0.0.1:8766"}' +inkcre-cli extension install yourname/notebook --version 0.1.0 +inkcre-cli extension enable yourname/notebook +inkcre-cli source types +``` + +Confirm `extensions.notebook.source.Source` appears. Save `notebook.json`, using the fixture URL +reachable from Core: + +```json +{ + "nickname": "Notebook test", + "config": { "url": "http://127.0.0.1:8765/note.json" } +} +``` + +```sh +inkcre-cli source create --type extensions.notebook.source.Source --input notebook.json +``` + +## 6. Verify behavior before distributing + +1. [Collect](/guide/collect) using the returned Source ID and wait for the Job to finish. +2. [Index and search](/guide/search) for `Notebook verification: blue heron`; read the text and + inspect its Source relation. +3. Collect the same fixture again. Confirm no second snapshot was added. +4. Change the fixture's text, collect again, and confirm a new snapshot is readable. +5. Make the fixture return invalid JSON. The Job must fail without advancing Source state; fix the + fixture and confirm a later run succeeds. + +This is a small runnable acceptance journey against your actual wheel and Host, not just an import +test. Do it on the disposable instance before inviting others to install your code. The example does +not provide complete multi-item reconciliation or guarantee ordering of overlapping fetches; avoid +overlapping runs and design that policy before using it as a multi-item collector. + +Stop any test Crons, disable/uninstall the test Extension, restore the prior Registry configuration +(delete the config only if it did not exist before), and stop the fixture/preview servers when done. +Removing the Extension does not itself erase collected graph data. For iteration, use a new release +version, disable all enabled Peers, and restart Core when replacing already-imported code; do not +overwrite published bytes or treat disable/re-enable as Python module reload. + +## Deliver your integration + +Keep the package in your own repository. To distribute through a Registry, obtain permission for +your namespace, prepare the exact release association, upload the finalized wheel, and publish the +release using the [Extension Toolkit](https://github.com/InKCre/ext-reg/tree/main/toolkit). +Operators then install your exact coordinate/version and follow your source-specific setup guide. +The static preview is a development path, not authorization to publish to `registry.inkcre.dev`. + +For larger collectors, study the +[RSS implementation](https://github.com/InKCre/core-py/tree/main/extensions/rss) for incremental +reconciliation and the +[Source runtime](https://github.com/InKCre/core-py/blob/main/app/business/source/main.py) for +current signatures. The +[native distribution contract](https://github.com/InKCre/core-py/blob/main/docs/40-deployment/native-extension-distribution.md) +owns Host installation, dependency admission, restart, and release rules. Pin and test the Host +compatibility you declare; a working example is not a promise that every `app.*` API is stable. diff --git a/website/content/en/developer/index.md b/website/content/en/developer/index.md index 6ab9b3e..a423971 100644 --- a/website/content/en/developer/index.md +++ b/website/content/en/developer/index.md @@ -9,7 +9,8 @@ InKCre is a multi-repository system organized around a shared info-base. This gu common mental model and routes you to the repository that owns the details. > [!IMPORTANT] InKCre is under active development. The contributor path is usable today, while -> third-party database, Extension, and API contracts are still being documented and may change. +> ecosystem interfaces remain version-sensitive. Check the Host compatibility of an Extension rather +> than assuming a stable, standalone SDK. ## Choose a path @@ -28,9 +29,12 @@ The foundational ecosystem path is participation as an authenticated peer of the Native PostgreSQL and PostgREST are transports over the same admitted, versioned database protocol. Extensions and APIs provide additional integration shapes. -This is not yet a promise of a stable public SDK, unrestricted database access, or a complete API -compatibility policy. The [Architecture guide](/developer/architecture#ecosystem-surfaces) explains -what is real now and where its canonical contracts live. +Start with [Ecosystem Developers](/developer/ecosystem/) to choose an integration path. To collect +from a new service, follow [Build a Source Extension](/developer/ecosystem/source-extension): +develop your own Python package and test it on a matching Core Host without contributing to Core. + +This is not a promise of unrestricted database access or a complete API compatibility policy. The +[Architecture guide](/developer/architecture) explains the wider model and canonical contracts. ## Primary repositories diff --git a/website/content/en/getting-started.md b/website/content/en/getting-started.md index a3bb8ac..27b1bef 100644 --- a/website/content/en/getting-started.md +++ b/website/content/en/getting-started.md @@ -61,10 +61,10 @@ personal account. ### 2. Collect one useful source -Begin with a public RSS feed you care about. The self-hosted walkthrough takes you through -installing its Extension, adding the Source configuration, and checking that the collection Job -completed. Once that works, add personal sources one at a time, with the credentials and permissions -each requires. +Choose [your first source](/guide/first-source); a public RSS feed is an easy starting point. Each +[source guide](/guide/sources) covers its own prerequisites, Extension setup, and first collection. +Once you can retrieve a known item, add personal sources one at a time, with the credentials and +permissions each requires. ### 3. Find and use what you collected diff --git a/website/content/en/guide/collect.md b/website/content/en/guide/collect.md new file mode 100644 index 0000000..4c71dc4 --- /dev/null +++ b/website/content/en/guide/collect.md @@ -0,0 +1,36 @@ +--- +title: Run a Collection +description: Execute one Source collection and distinguish its Source ID from its Job ID. +--- + +# Run a Collection + +First create a Source using [its setup guide](/guide/sources). Keep the returned Source ID. + +1. Replace `42` with that Source ID and run: + + ```sh + inkcre-cli source collect 42 --input-json '{}' + ``` + + The empty object uses that collector's default run options. Source-specific options belong to the + individual guide; they are not a replacement for the Source's saved account configuration. + +2. The response contains a new **Job ID**. Replace `17` with it: + + ```sh + inkcre-cli job wait 17 --for 30s + ``` + +3. Look for `status: finished`. If still `pending` or `running`, observe the same Job again. Ending + observation does not cancel it. If it failed, inspect: + + ```sh + inkcre-cli job get 17 --json + ``` + +Read source-specific diagnostics even after success. Do not repeatedly create Jobs to work around a +provider error or an observation timeout; fix the reported cause first. Review output for private +information before sharing it. + +Next: [index and search for a known item](/guide/search), then [schedule](/guide/schedules). diff --git a/website/content/en/guide/first-source.md b/website/content/en/guide/first-source.md index ba859e4..f551136 100644 --- a/website/content/en/guide/first-source.md +++ b/website/content/en/guide/first-source.md @@ -1,76 +1,36 @@ --- title: Collect Your First Source -description: Install the RSS collector and collect your first feed. +description: Choose a source and complete your first collection and search. --- # Collect Your First Source -Before you start, [connect the CLI](/guide/connect-cli) to your instance. +Start with a working instance and a [connected CLI](/guide/connect-cli). You do not need to connect +every account at once: choose one small source with an item you will recognize. -Choose a publication you already read and copy its **RSS or Atom feed URL**. Its normal homepage URL -is not usually a feed URL. Look for an RSS/Subscribe link on the publication. +## Choose your first source -## Enable the collector +[RSS or Atom](/guide/sources/rss) is a useful first choice because a public feed needs no account +credentials. If you prefer your own saved information, start with +[GitHub Stars](/guide/sources/github), [email](/guide/sources/mail), +[Telegram](/guide/sources/telegram), or [Twitter / X bookmarks](/guide/sources/twitter). Each guide +includes its own prerequisites and setup. -Install the RSS Extension on Core, then enable it: +An **Extension** supplies a collector implementation; a **Source** is one configured use of it. For +example, install the RSS Extension once, then create one Source per feed. Installing it does not +automatically enable it or collect anything. Enable the collector on Core, not only in the Web app. -```sh -inkcre-cli extension install inkcre/rss --version 0.2.0 -inkcre-cli extension enable inkcre/rss -inkcre-cli source types -``` +## Complete the first loop -Version `0.2.0` is a published Python release for Core Host `0.2.x`. For a different Host version, -check the [RSS release listing](https://registry.inkcre.dev/v1/extensions/inkcre/rss) for a -published release with a compatible `python.host_sdk_version`. Do not guess a version or use -`latest`. If already installed, inspect `inkcre-cli extension get inkcre/rss` before changing it. +1. Follow your chosen source's guide to enable its Extension and create the Source. Retain the + returned **Source ID**. +2. [Run a Collection](/guide/collect) and wait for the returned **Job ID** to finish. These are + different IDs: the Source persists across runs; each Job represents one run. +3. [Find What You Saved](/guide/search): maintain the lexical index and search for a known item. +4. Only after that works, [schedule collection and indexing](/guide/schedules). -The type list should contain `extensions.rss.rss.Source` and `extensions.rss.atom.Source`. The Web -app's Extensions switch controls its browser runtime; it does not enable this Core collector. +**Done means you retrieved a real item**, not just that installation or a Job succeeded. A +successful empty collection may be normal; each source guide explains what is eligible for +collection. -## Add and run the source - -1. Create `feed.json` in your local working folder: - - ```json - { - "nickname": "My first feed", - "config": { - "feed_url": "https://YOUR-PUBLICATION/FEED", - "fetch_full_text": false, - "download_enclosures": false - } - } - ``` - - Replace the URL. These first-run settings collect feed content without extra article fetches or - attachment downloads. - -2. Create the Source, choosing the matching feed format: - - ```sh - inkcre-cli source create --type extensions.rss.rss.Source --input feed.json - ``` - - For Atom, use `extensions.rss.atom.Source`. Record the returned Source `id`. - -3. Collect it. Replace `42` with your Source ID: - - ```sh - inkcre-cli source collect 42 --input-json '{}' - ``` - -4. The response contains a **Job ID**, different from the Source ID. Replace `17` with it: - - ```sh - inkcre-cli job wait 17 --for 30s - ``` - - Look for `status: finished`. If still `pending` or `running`, observe again rather than creating - another collect job. If `failed`, inspect `inkcre-cli job get 17 --json` and its diagnostics. - Ending observation does not stop the Job. - -**Checkpoint:** the collection Job finished. A feed exposes only what its publisher currently -provides; success does not mean its entire historical archive was imported. - -Next: [Find what you saved](/guide/search). +Next: [Connect More Sources](/guide/sources), or [Use Your Information](/guide/daily-use). diff --git a/website/content/en/guide/search.md b/website/content/en/guide/search.md index 412bcaa..161c56e 100644 --- a/website/content/en/guide/search.md +++ b/website/content/en/guide/search.md @@ -19,10 +19,10 @@ Use its returned Job ID with `inkcre-cli job wait JOB_ID --for 30s`, replacing ` number. Check its status and `state` for indexed records and diagnostics. Maintenance processes a bounded batch; repeat if you imported more than one batch can index. -Copy a distinctive phrase from an item in your feed and search: +Copy a distinctive phrase from an item your source collected and search: ```sh -inkcre-cli recall 'A phrase from your feed item' --mode lexical +inkcre-cli recall 'A phrase from your saved item' --mode lexical ``` Read a returned Block's content and relationships, replacing `123` with that Block's ID: diff --git a/website/content/en/guide/sources.md b/website/content/en/guide/sources.md index 75145b1..85ea93f 100644 --- a/website/content/en/guide/sources.md +++ b/website/content/en/guide/sources.md @@ -1,145 +1,42 @@ --- title: Connect More Sources -description: Connect GitHub, email, Telegram, and other supported sources. +description: Choose an independent setup guide for each information source. --- # Connect More Sources -These instructions use an already [connected CLI](/guide/connect-cli). Follow the -[first-source walkthrough](/guide/first-source) once to learn how Source IDs and collection Jobs -work. +Use the guide for the information you want to bring into InKCre. Each page starts from a working +instance and a [connected CLI](/guide/connect-cli); you do not need to complete the RSS tutorial +first. -For each new source: enable its Core Extension, create the Source, collect once, inspect the Job, -run index maintenance, and search for a known item. Only then add a schedule. +| What you want to collect | Setup guide | What you need | +| -------------------------------------------- | ------------------------------------------------ | ---------------------------------------- | +| Articles from a publication | [RSS and Atom](/guide/sources/rss) | A feed URL | +| Saved repositories and Lists | [GitHub Stars and Lists](/guide/sources/github) | A GitHub personal access token | +| New or historical mail | [Email over IMAP](/guide/sources/mail) | IMAP access and supported credentials | +| Text and messages you forward | [Telegram inbox](/guide/sources/telegram) | A dedicated bot and your numeric user ID | +| Your bookmarked posts | [Twitter / X bookmarks](/guide/sources/twitter) | An X OAuth app and API access | +| Notes you write in a Memos-compatible client | [Memos-compatible capture](/guide/sources/memos) | A supported client and a dedicated PAT | -The versions below are published releases for Core Host `0.2.x`. Check the linked release listings -and your running type's `--schema` when versions differ. Source files and command output may contain -account credentials; do not publish them. +Memos is a write-in capture interface, not a collector that imports an existing Memos server. An +arbitrary webpage, social account, or service is not automatically a supported Source. -## GitHub Stars and Lists +## Add one source at a time -1. Create a personal access token for the account whose Stars and Lists you want to collect, - following - [GitHub's token instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens). - Its account and permissions determine which data is visible. -2. Enable the Extension: +Install and enable its Core Extension, complete any account authorization, create the Source, then +[collect once](/guide/collect). [Index and search](/guide/search) for an item you recognize before +[scheduling](/guide/schedules). More successful Jobs do not by themselves prove that more +information was imported. - ```sh - inkcre-cli extension install inkcre/github --version 0.3.0 - inkcre-cli extension enable inkcre/github - ``` +The guides pin published releases for Core Host `0.2.x`. Inspect existing installations before +changing versions and consult the linked Registry release listing for compatibility. Keep local +configuration files, tokens, and management-command output private. -3. Save `github.json` with your token: +## Build a missing collector - ```json - { - "nickname": "My GitHub saves", - "config": { "github_token": "YOUR-GITHUB-TOKEN" } - } - ``` +An Extension can add a Source without becoming part of the Core repository. If you know Python and +want to connect another service, follow +[Build a Source Extension](/developer/ecosystem/source-extension). That is the ecosystem developer +path; changing InKCre itself has a separate [contributor guide](/developer/contributing). - ```sh - inkcre-cli source create --type extensions.github.stars.Source --input github.json - ``` - -4. Collect the returned Source ID using the [first-source procedure](/guide/first-source). After - indexing, search for a repository you starred. Each run synchronizes current Stars and Lists; it - is not a code backup or a full GitHub activity archive. - -See the [GitHub collector](https://github.com/InKCre/core-py/blob/main/extensions/github/README.md) -and [published releases](https://registry.inkcre.dev/v1/extensions/inkcre/github). - -## Email over IMAP - -1. Find your provider's IMAP hostname and enable IMAP if required. Obtain an app-specific password - where supported. Accounts requiring an unsupported authentication method cannot be connected by - substituting an ordinary login password. -2. Install and enable `inkcre/mail` version `0.3.0`, using the two Extension commands above with - `inkcre/mail` in place of `inkcre/github`. -3. Save `mail.json`, replacing the host and credentials: - - ```json - { - "nickname": "My mail", - "config": { - "protocol": "imap", - "parameters": { - "host": "YOUR-IMAP-HOST", - "port": 993, - "security": "tls", - "username": "YOUR-MAIL-LOGIN", - "password": "YOUR-APP-PASSWORD" - }, - "ordinary_mark_as_seen": false, - "synchronize_deletions": false - } - } - ``` - - ```sh - inkcre-cli source create --type extensions.mail.source.Source --input mail.json - ``` - -4. Send yourself a test email **after creating the Source**, then collect its ID. Ordinary - collection starts with new mail; an empty first run does not imply login failed. Inspect mailbox - diagnostics even if the Job finishes. This example preserves unread state; the collector's - default would mark ordinary collected mail as seen. -5. For older mail, request a small explicit backfill. Replace `42` with the Mail Source ID and - choose dates for your mailbox: - - ```sh - inkcre-cli source backfill 42 --input-json '{"since":"2026-09-01","before":"2026-09-08"}' - ``` - - The start date is included and the end date excluded. Observe the Job and index afterward. This - collector does not send mail or create email digests. - -See the -[Mail configuration](https://github.com/InKCre/core-py/blob/main/docs/30-unit-tdd/mail-extension.md) -and [published releases](https://registry.inkcre.dev/v1/extensions/inkcre/mail). - -## Send or forward messages from Telegram - -1. Create a dedicated bot with [@BotFather](https://t.me/botfather): send `/newbot`, follow its - naming prompts, and retain the token. Obtain your own numeric user ID using - [Telegram Desktop's data export](https://telegram.org/blog/export-and-more): open **Settings → - Advanced → Export Telegram data**, include personal information, and choose JSON format. In the - exported JSON, use `personal_information.user_id`, not your `@username` or the bot's ID. Telegram - may require confirmation from another device or a waiting period. -2. Install and enable `inkcre/telegram` version `0.3.0` with the same Extension commands. -3. Save `telegram.json`, replacing the token and example user ID: - - ```json - { - "nickname": "My Telegram inbox", - "config": { - "bot_token": "YOUR-BOT-TOKEN", - "bound_user_id": 123456789, - "download_attachments": false - } - } - ``` - - ```sh - inkcre-cli source create --type extensions.telegram.source.Source --input telegram.json - ``` - -4. Send a distinctive text message in a private chat with the bot, then collect the Source ID. - Successfully saved messages receive a 👍 reaction. After indexing, search for its text. -5. Add a schedule to forward messages without running the CLI each time. `*/5 * * * *` polls every - five minutes while Core is awake. Telegram retains updates for a limited time, so a long-sleeping - instance should not be your only copy. - -Use one bot for one Source. This is a capture inbox, not group/channel history import or a -notification destination. Attachments are metadata-only in this example; enable -`download_attachments` when you want their bytes saved too. See the -[Telegram collector](https://github.com/InKCre/core-py/blob/main/extensions/telegram/README.md) and -[published releases](https://registry.inkcre.dev/v1/extensions/inkcre/telegram). - -Add more RSS/Atom subscriptions with one Source per feed. Memos-compatible capture is also available -through a separately configured -[Memos Extension](https://github.com/InKCre/core-py/blob/main/docs/30-unit-tdd/memos-extension.md). -An arbitrary service or browser account is not automatically a supported Source; check for a -collector before assuming its history can be imported. - -Next: [Use your information](/guide/daily-use). +Next: [Use Your Information](/guide/daily-use). diff --git a/website/content/en/guide/sources/github.md b/website/content/en/guide/sources/github.md new file mode 100644 index 0000000..197b525 --- /dev/null +++ b/website/content/en/guide/sources/github.md @@ -0,0 +1,45 @@ +--- +title: GitHub Stars and Lists +description: Connect your GitHub saves to InKCre. +--- + +# GitHub Stars and Lists + +Start with a working instance and a [connected CLI](/guide/connect-cli). The version below targets +Core Host `0.2.x`; check the linked release listing for other Host versions. If already installed, +inspect `inkcre-cli extension get inkcre/github` before changing it. Keep credential files and +command output private. + +1. Create a personal access token for the account whose Stars and Lists you want to collect, + following + [GitHub's token instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens). + Its account and permissions determine which data is visible. +2. Enable the Extension: + + ```sh + inkcre-cli extension install inkcre/github --version 0.3.0 + inkcre-cli extension enable inkcre/github + ``` + +3. Save `github.json` with your token: + + ```json + { + "nickname": "My GitHub saves", + "config": { "github_token": "YOUR-GITHUB-TOKEN" } + } + ``` + + ```sh + inkcre-cli source create --type extensions.github.stars.Source --input github.json + ``` + +4. Collect the returned Source ID using [Run a Collection](/guide/collect). After indexing, search + for a repository you starred. Each run synchronizes current Stars and Lists; it is not a code + backup or a full GitHub activity archive. + +See the [GitHub collector](https://github.com/InKCre/core-py/blob/main/extensions/github/README.md) +and [published releases](https://registry.inkcre.dev/v1/extensions/inkcre/github). + +Next: [index and search for a known item](/guide/search), then +[schedule collection](/guide/schedules). diff --git a/website/content/en/guide/sources/mail.md b/website/content/en/guide/sources/mail.md new file mode 100644 index 0000000..61145ac --- /dev/null +++ b/website/content/en/guide/sources/mail.md @@ -0,0 +1,66 @@ +--- +title: Email over IMAP +description: Connect your IMAP mailbox to InKCre. +--- + +# Email over IMAP + +Start with a working instance and a [connected CLI](/guide/connect-cli). The version below targets +Core Host `0.2.x`; check the linked release listing for other Host versions. If already installed, +inspect `inkcre-cli extension get inkcre/mail` before changing it. Keep credential files and command +output private. + +1. Find your provider's IMAP hostname and enable IMAP if required. Obtain an app-specific password + where supported. Accounts requiring an unsupported authentication method cannot be connected by + substituting an ordinary login password. +2. Install and enable the Core Extension: + + ```sh + inkcre-cli extension install inkcre/mail --version 0.3.0 + inkcre-cli extension enable inkcre/mail + ``` + +3. Save `mail.json`, replacing the host and credentials: + + ```json + { + "nickname": "My mail", + "config": { + "protocol": "imap", + "parameters": { + "host": "YOUR-IMAP-HOST", + "port": 993, + "security": "tls", + "username": "YOUR-MAIL-LOGIN", + "password": "YOUR-APP-PASSWORD" + }, + "ordinary_mark_as_seen": false, + "synchronize_deletions": false + } + } + ``` + + ```sh + inkcre-cli source create --type extensions.mail.source.Source --input mail.json + ``` + +4. Send yourself a test email **after creating the Source**, then [collect its ID](/guide/collect). + Ordinary collection starts with new mail; an empty first run does not imply login failed. Inspect + mailbox diagnostics even if the Job finishes. This example preserves unread state; the + collector's default would mark ordinary collected mail as seen. +5. For older mail, request a small explicit backfill. Replace `42` with the Mail Source ID and + choose dates for your mailbox: + + ```sh + inkcre-cli source backfill 42 --input-json '{"since":"2026-09-01","before":"2026-09-08"}' + ``` + + The start date is included and the end date excluded. Observe the Job and index afterward. This + collector does not send mail or create email digests. + +See the +[Mail configuration](https://github.com/InKCre/core-py/blob/main/docs/30-unit-tdd/mail-extension.md) +and [published releases](https://registry.inkcre.dev/v1/extensions/inkcre/mail). + +Next: [index and search for a known item](/guide/search), then +[schedule collection](/guide/schedules). diff --git a/website/content/en/guide/sources/memos.md b/website/content/en/guide/sources/memos.md new file mode 100644 index 0000000..9bd8e7b --- /dev/null +++ b/website/content/en/guide/sources/memos.md @@ -0,0 +1,67 @@ +--- +title: Memos-Compatible Capture +description: Write notes into InKCre from a supported Memos client. +--- + +# Memos-Compatible Capture + +This is a capture endpoint for notes you write, **not** an importer for an existing Memos server. It +implements a bounded Memos `0.29.1` API subset; the previously accepted client is MoeMemos Android +`2.0.4`. Other clients or versions may use unsupported endpoints. + +## 1. Configure the Extension + +Start with a [connected CLI](/guide/connect-cli) and a Core HTTPS URL. Install the Core Host `0.2.x` +release, or inspect `inkcre-cli extension get inkcre/memos` if already installed: + +```sh +inkcre-cli extension install inkcre/memos --version 0.2.0 +``` + +Generate a dedicated PAT and save it in your password manager. Its required format is `memos_pat_` +followed by exactly 32 ASCII letters or digits. For example, run locally: + +```sh +python -c 'import secrets, string; print("memos_pat_" + "".join(secrets.choice(string.ascii_letters + string.digits) for _ in range(32)))' +``` + +In a private folder, save `memos.json`, replacing the placeholder with that PAT: + +```json +{ + "personal_access_token": "YOUR-GENERATED-MEMOS-PAT" +} +``` + +```sh +inkcre-cli extension config update inkcre/memos --input memos.json +inkcre-cli extension enable inkcre/memos +``` + +The update response can contain the PAT; keep both it and the file private. This token authorizes +the Memos interface, not general Core administration. Never substitute `JWT_SECRET`. + +## 2. Connect the client and write a note + +1. In the supported client's server/account setup, use `https://YOUR-CORE-HOST/memos` as the server + address and the PAT as its access token. Do not use your PostgREST URL or append `/api/v1`. +2. Write a short note with distinctive text, such as “Memos capture verification orchid”. +3. Refresh the client's list and reopen the note to confirm it was saved. +4. [Maintain the index and search](/guide/search) for the same text from InKCre. + +There is no `source create` or collection Cron for this path: saving in the client sends the note +directly to InKCre. Schedule index maintenance if you want new notes searchable without a manual +run. The supported subset includes notes, attachments, and comments, not full Memos administration, +social features, or browsing the entire InKCre info-base. + +## Disconnect or troubleshoot + +If login fails, verify the `/memos` server prefix, PAT format, and running Core Extension. An +unsupported client endpoint can fail even with valid credentials; use the pinned client/API pairing +before diagnosing the instance itself. + +Remove the account from the client and rotate or clear `personal_access_token` to revoke the old +credential. Disable the Extension to withdraw its routes. These actions do not erase stored notes. + +See the [release listing](https://registry.inkcre.dev/v1/extensions/inkcre/memos) and the +[Memos runtime contract](https://github.com/InKCre/core-py/blob/main/docs/30-unit-tdd/memos-extension.md). diff --git a/website/content/en/guide/sources/rss.md b/website/content/en/guide/sources/rss.md new file mode 100644 index 0000000..2f0f5aa --- /dev/null +++ b/website/content/en/guide/sources/rss.md @@ -0,0 +1,63 @@ +--- +title: RSS and Atom +description: Install the RSS collector and collect your first feed. +--- + +# RSS and Atom + +Before you start, [connect the CLI](/guide/connect-cli) to your instance. + +Choose a publication you already read and copy its **RSS or Atom feed URL**. Its normal homepage URL +is not usually a feed URL. Look for an RSS/Subscribe link on the publication. + +## Enable the collector + +Install the RSS Extension on Core, then enable it: + +```sh +inkcre-cli extension install inkcre/rss --version 0.2.0 +inkcre-cli extension enable inkcre/rss +inkcre-cli source types +``` + +Version `0.2.0` is a published Python release for Core Host `0.2.x`. For a different Host version, +check the [RSS release listing](https://registry.inkcre.dev/v1/extensions/inkcre/rss) for a +published release with a compatible `python.host_sdk_version`. Do not guess a version or use +`latest`. If already installed, inspect `inkcre-cli extension get inkcre/rss` before changing it. + +The type list should contain `extensions.rss.rss.Source` and `extensions.rss.atom.Source`. The Web +app's Extensions switch controls its browser runtime; it does not enable this Core collector. + +## Add and run the source + +1. Create `feed.json` in your local working folder: + + ```json + { + "nickname": "My first feed", + "config": { + "feed_url": "https://YOUR-PUBLICATION/FEED", + "fetch_full_text": false, + "download_enclosures": false + } + } + ``` + + Replace the URL. These first-run settings collect feed content without extra article fetches or + attachment downloads. + +2. Create the Source, choosing the matching feed format: + + ```sh + inkcre-cli source create --type extensions.rss.rss.Source --input feed.json + ``` + + For Atom, use `extensions.rss.atom.Source`. Record the returned Source `id`. + +3. [Run a Collection](/guide/collect) using that Source ID. The guide shows the collection command, + how to wait for its separate Job ID, and what to inspect if it fails. + +**Checkpoint:** the collection Job finished. A feed exposes only what its publisher currently +provides; success does not mean its entire historical archive was imported. + +Next: [Find what you saved](/guide/search), then [schedule collection](/guide/schedules). diff --git a/website/content/en/guide/sources/telegram.md b/website/content/en/guide/sources/telegram.md new file mode 100644 index 0000000..0be4b17 --- /dev/null +++ b/website/content/en/guide/sources/telegram.md @@ -0,0 +1,57 @@ +--- +title: Send or forward messages from Telegram +description: Connect a personal Telegram capture inbox to InKCre. +--- + +# Send or forward messages from Telegram + +Start with a working instance and a [connected CLI](/guide/connect-cli). The version below targets +Core Host `0.2.x`; check the linked release listing for other Host versions. If already installed, +inspect `inkcre-cli extension get inkcre/telegram` before changing it. Keep credential files and +command output private. + +1. Create a dedicated bot with [@BotFather](https://t.me/botfather): send `/newbot`, follow its + naming prompts, and retain the token. Obtain your own numeric user ID using + [Telegram Desktop's data export](https://telegram.org/blog/export-and-more): open **Settings → + Advanced → Export Telegram data**, include personal information, and choose JSON format. In the + exported JSON, use `personal_information.user_id`, not your `@username` or the bot's ID. Telegram + may require confirmation from another device or a waiting period. +2. Install and enable the Core Extension: + + ```sh + inkcre-cli extension install inkcre/telegram --version 0.3.0 + inkcre-cli extension enable inkcre/telegram + ``` + +3. Save `telegram.json`, replacing the token and example user ID: + + ```json + { + "nickname": "My Telegram inbox", + "config": { + "bot_token": "YOUR-BOT-TOKEN", + "bound_user_id": 123456789, + "download_attachments": false + } + } + ``` + + ```sh + inkcre-cli source create --type extensions.telegram.source.Source --input telegram.json + ``` + +4. Send a distinctive text message in a private chat with the bot, then + [collect the Source ID](/guide/collect). Successfully saved messages receive a 👍 reaction. After + indexing, search for its text. +5. Add a schedule to forward messages without running the CLI each time. `*/5 * * * *` polls every + five minutes while Core is awake. Telegram retains updates for a limited time, so a long-sleeping + instance should not be your only copy. + +Use one bot for one Source. This is a capture inbox, not group/channel history import or a +notification destination. Attachments are metadata-only in this example; enable +`download_attachments` when you want their bytes saved too. See the +[Telegram collector](https://github.com/InKCre/core-py/blob/main/extensions/telegram/README.md) and +[published releases](https://registry.inkcre.dev/v1/extensions/inkcre/telegram). + +Next: [index and search for a known item](/guide/search), then +[schedule collection](/guide/schedules). diff --git a/website/content/en/guide/sources/twitter.md b/website/content/en/guide/sources/twitter.md new file mode 100644 index 0000000..0c135fc --- /dev/null +++ b/website/content/en/guide/sources/twitter.md @@ -0,0 +1,159 @@ +--- +title: Twitter / X Bookmarks +description: Authorize an X account and collect its bookmarked posts. +--- + +# Twitter / X Bookmarks + +This guide uses the official X API to collect your own bookmarks. Start with a +[connected CLI](/guide/connect-cli), a Core Host `0.2.x` instance with a public HTTPS URL, and an X +account containing a recent bookmark whose text you recognize. + +You also need an X developer app with OAuth 2.0 user authorization and access to the required API +endpoints. API access can incur charges; check your app's access and billing before starting. The +[X Bookmarks documentation](https://docs.x.com/x-api/posts/bookmarks/introduction) owns provider +requirements. An app-only bearer token is not a substitute for authorizing your account. + +## 1. Enable the Core Extension + +```sh +inkcre-cli extension install inkcre/twitter --version 0.4.0 +inkcre-cli extension enable inkcre/twitter +inkcre-cli source types +``` + +Look for `extensions.twitter.bookmark.Source`. If already installed, inspect +`inkcre-cli extension get inkcre/twitter` before changing it. Check the +[release listing](https://registry.inkcre.dev/v1/extensions/inkcre/twitter) for other Host versions. +This procedure does not require a browser-side Twitter Extension. + +## 2. Connect your X account + +Authorization belongs to the **Extension**, not an individual Source. All Twitter bookmark Sources +in this deployment use the connected account. Do not switch accounts to create a second user's +Source: that changes the account used by existing Sources too. + +The CLI does not expose this Extension's OAuth setup commands. The following local script calls its +authenticated setup API. Use the Python environment from [Connect the CLI](/guide/connect-cli), +which already supplies `httpx` and `PyJWT`. If someone else operates Core, ask them to perform +setup; do not ask them to share their signing secret. + +Save this as `connect-twitter.py`, then run `python connect-twitter.py`. It first prints the exact +callback URL. In your X app's user authentication settings, enable OAuth 2.0, choose a confidential +web-app client that provides a Client ID and Client Secret, and register that callback URL exactly. +Keep the script open while doing this. Use the OAuth Client ID/Secret, not API Key/Secret. See +[X's OAuth app settings](https://docs.x.com/fundamentals/authentication/oauth-2-0/authorization-code) +if you cannot find those fields. + +```python +import getpass +import time + +import httpx +import jwt + +core_url = input("Core HTTPS URL: ").strip().rstrip("/") +if not core_url.startswith("https://"): + raise ValueError("Use your trusted Core HTTPS URL, not PostgREST") +secret = getpass.getpass("InKCre JWT_SECRET: ") +if not secret: + raise ValueError("JWT_SECRET is required") + +with httpx.Client(timeout=30) as client: + def request(method, path, **kwargs): + issued = int(time.time()) - 5 + token = jwt.encode( + {"role": "authenticated", "iss": "inkcre-peer", "aud": "inkcre-api", + "iat": issued, "exp": issued + 900}, + secret, algorithm="HS256", + ) + response = client.request( + method, core_url + path, + headers={"Authorization": f"Bearer {token}"}, **kwargs, + ) + response.raise_for_status() + return response.json() + + status = request("GET", "/twitter/setup") + print("Register this callback URL in X:", status["callback_url"]) + if not status["connected"]: + client_id = getpass.getpass("X OAuth Client ID: ").strip() + client_secret = getpass.getpass("X OAuth Client Secret: ") + if not client_id or not client_secret: + raise ValueError("Both OAuth app credentials are required") + request("PUT", "/twitter/setup/oauth-app", json={ + "client_id": client_id, "client_secret": client_secret, + }) + transaction = request("POST", "/twitter/setup/oauth-transactions") + print("Open this private authorization URL:", transaction["authorize_url"]) + input("Authorize in X; after the callback says connected, press Enter: ") + result = request("POST", "/twitter/setup/oauth-transaction", json={ + "transaction_id": transaction["id"], + }) + if result["status"] != "succeeded": + raise RuntimeError(f"Authorization did not complete: {result['status']}") + status = request("GET", "/twitter/setup") + if not status["connected"]: + raise RuntimeError("Account is not connected; inspect setup status") + print("Connected account:", status["handle"]) +``` + +Approve the account you intend to collect. The Extension requests `tweet.read`, `users.read`, +`bookmark.read`, and `offline.access` so it can refresh authorization. Complete the browser flow +within ten minutes. Keep credentials and the authorization URL private; never enter `JWT_SECRET` +into X. Account tokens stay in the deployment, whose operator and admitted Peers are trusted. + +If the callback URL is missing or wrong, fix the Core Peer's `http_public_base_url` before +proceeding; it must identify Core's public HTTPS origin, yielding `/twitter/auth/callback`. Starting +a new authorization supersedes the previous pending attempt. A request to replace an already +configured OAuth app may return HTTP 409: do not blindly enable `confirm_account_reset`, because +replacement disconnects the existing account. + +## 3. Create and collect a Source + +Save `twitter.json`: + +```json +{ + "nickname": "My X bookmarks", + "config": {} +} +``` + +```sh +inkcre-cli source create --type extensions.twitter.bookmark.Source --input twitter.json +``` + +Record the returned Source ID. Replace `42` with it and collect: + +```sh +inkcre-cli source collect 42 --input-json '{"result_limit":40}' +``` + +Use the returned Job ID with [Run a Collection](/guide/collect), then +[index and search](/guide/search) for text from a recent bookmark. Once that works, add a +[schedule](/guide/schedules). + +**Scope:** an ordinary run reads one page, with `result_limit` from 5 to 100 (default 40), stopping +at the previously seen bookmark when present. This is not a guaranteed archive of all bookmarks, +folders, replies, or unbookmarks. The current `full` option also performs a single page fetch per +Job and does not automatically drain the entire history; do not treat it as a complete backfill. +Collect often enough for your usage and retain another copy of information you cannot afford to +lose. + +## If setup or collection fails + +- **404 on setup:** check the Core URL and whether this Extension is running on that Core Peer. +- **401:** check your InKCre connection for setup failures; reconnect X if the collection's provider + authorization has expired or been revoked. +- **402/403 from X:** inspect app access, credits, permissions, and the authorized account. +- **429:** respect the provider's rate limit instead of repeatedly submitting Jobs. +- **Finished but no new items:** check the connected handle and bookmark a new recognizable post. + +To stop collection, disable its Cron first. An operator can disconnect the stored account using +authenticated `DELETE /twitter/setup/account` with the same request pattern above, and separately +revoke the app in X's account settings. Disconnecting does not delete previously collected data. The +alternate `twikit` backend exists, but its account-login mechanics are not this OAuth walkthrough. + +Implementation reference: +[Twitter Extension](https://github.com/InKCre/core-py/tree/main/extensions/twitter). diff --git a/website/content/en/self-hosted/getting-started.md b/website/content/en/self-hosted/getting-started.md index a9be917..f40b8d9 100644 --- a/website/content/en/self-hosted/getting-started.md +++ b/website/content/en/self-hosted/getting-started.md @@ -29,7 +29,8 @@ Core URL, a PostgREST URL, your private JWT secret, and the Core Peer ID. Follow these shared user guides in order. If you already have an instance, start here. 1. [Connect the CLI](/guide/connect-cli) and verify authenticated access. -2. [Collect your first source](/guide/first-source), using one public RSS or Atom feed. +2. [Collect your first source](/guide/first-source); a public RSS or Atom feed is a simple starting + point. 3. [Find what you saved](/guide/search): maintain the lexical index, search, and inspect a result. **Checkpoint:** you can retrieve a real item from your source. Collection and indexing are separate; From dc97b3712a8170fff3a3a52b717c2293e4769a6c Mon Sep 17 00:00:00 2001 From: Lan_zhijiang Date: Sun, 20 Sep 2026 17:20:28 +0800 Subject: [PATCH 08/18] =?UTF-8?q?docs(guides):=20=E4=BC=98=E5=85=88?= =?UTF-8?q?=E5=B1=95=E7=A4=BA=20Web=20=E6=93=8D=E4=BD=9C=E5=B9=B6=E6=8F=90?= =?UTF-8?q?=E4=BE=9B=20Agent=20=E6=8C=87=E5=8D=97=E5=88=87=E6=8D=A2?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 补齐 Twitter setup wizard 与六种来源的可视化步骤 - 共享连接和扩展准备入口，修正浏览器自动注册说明 - 明确安装、索引维护及 Twitter 双端发行兼容边界 --- website/.vitepress/config.mts | 4 +- website/.vitepress/theme/InterfaceGuide.vue | 54 ++++++++++++++ website/.vitepress/theme/index.ts | 11 ++- website/README.md | 9 +++ website/content/en/getting-started.md | 4 +- website/content/en/guide/collect.md | 31 ++++++++ website/content/en/guide/connect-cli.md | 5 +- website/content/en/guide/connect.md | 52 +++++++++++++ website/content/en/guide/daily-use.md | 44 +++-------- website/content/en/guide/extensions.md | 60 +++++++++++++++ website/content/en/guide/first-source.md | 13 ++-- website/content/en/guide/schedules.md | 42 +++++++++++ website/content/en/guide/search.md | 40 ++++++++++ website/content/en/guide/sources.md | 7 +- website/content/en/guide/sources/github.md | 34 +++++++++ website/content/en/guide/sources/mail.md | 52 +++++++++++++ website/content/en/guide/sources/memos.md | 40 ++++++++++ website/content/en/guide/sources/rss.md | 44 ++++++++++- website/content/en/guide/sources/telegram.md | 44 +++++++++++ website/content/en/guide/sources/twitter.md | 73 +++++++++++++++++-- .../content/en/self-hosted/getting-started.md | 9 ++- 21 files changed, 616 insertions(+), 56 deletions(-) create mode 100644 website/.vitepress/theme/InterfaceGuide.vue create mode 100644 website/content/en/guide/connect.md create mode 100644 website/content/en/guide/extensions.md diff --git a/website/.vitepress/config.mts b/website/.vitepress/config.mts index 151da2d..02c527f 100644 --- a/website/.vitepress/config.mts +++ b/website/.vitepress/config.mts @@ -60,7 +60,9 @@ export default defineConfig({ { text: 'User Guide', items: [ - { text: 'Connect the CLI', link: '/guide/connect-cli' }, + { text: 'Connect to Your Instance', link: '/guide/connect' }, + { text: 'CLI / Agent Connection', link: '/guide/connect-cli' }, + { text: 'Prepare an Extension', link: '/guide/extensions' }, { text: 'Collect Your First Source', link: '/guide/first-source' }, { text: 'Find What You Saved', link: '/guide/search' }, { text: 'Schedule Collection and Indexing', link: '/guide/schedules' }, diff --git a/website/.vitepress/theme/InterfaceGuide.vue b/website/.vitepress/theme/InterfaceGuide.vue new file mode 100644 index 0000000..9fed22c --- /dev/null +++ b/website/.vitepress/theme/InterfaceGuide.vue @@ -0,0 +1,54 @@ + + + + + diff --git a/website/.vitepress/theme/index.ts b/website/.vitepress/theme/index.ts index 42fe9a9..9ba1962 100644 --- a/website/.vitepress/theme/index.ts +++ b/website/.vitepress/theme/index.ts @@ -1,4 +1,13 @@ import DefaultTheme from 'vitepress/theme' +import type { EnhanceAppContext } from 'vitepress' +import { ref } from 'vue' +import InterfaceGuide from './InterfaceGuide.vue' import './custom.css' -export default DefaultTheme +export default { + extends: DefaultTheme, + enhanceApp({ app }: EnhanceAppContext) { + app.provide('guide-interface', ref('web')) + app.component('InterfaceGuide', InterfaceGuide) + }, +} diff --git a/website/README.md b/website/README.md index 3048ed5..1b8d8ec 100644 --- a/website/README.md +++ b/website/README.md @@ -52,6 +52,15 @@ are resolved from the rewritten route, not the source file location. ## Page Authoring +- User procedures default to client-web. Use the shared `InterfaceGuide` component with `#web` and + `#cli` slots for alternate steps on the same route; CLI instructions primarily serve Agents and + operators. The choice survives client-side navigation, not a full reload. Without JavaScript, both + sections remain readable. Keep shared prerequisites and limitations outside the slots. +- Set `outline: false` on interface-switching pages: the default VitePress outline includes hidden + slot headings. Do not expose links to invisible instructions. The site sidebar remains available. +- State actual interface gaps instead of implying feature parity. Core package installation and + lexical maintenance still need CLI/operator steps; a browser wizard requires a compatible native + distribution as well as its Core collector. Check both against published Registry releases. - Keep exactly one H1 per page. - Add a concise page `description` in frontmatter. - Give headings explicit custom anchors only when another page or external consumer needs a durable diff --git a/website/content/en/getting-started.md b/website/content/en/getting-started.md index 27b1bef..eec100f 100644 --- a/website/content/en/getting-started.md +++ b/website/content/en/getting-started.md @@ -51,8 +51,8 @@ does not create an instance for you. first collection. [Advanced](/self-hosted/advanced) covers manual deployment and operating it on infrastructure you choose. - **Already have access to an instance:** obtain connection details from the person operating it, - then follow [Connect the CLI](/guide/connect-cli), [collect a source](/guide/first-source), and - [connect your everyday tools](/guide/daily-use). Skip the deployment steps. Only connect + then [connect the Web app or your Agent](/guide/connect), [collect a source](/guide/first-source), + and [connect your everyday tools](/guide/daily-use). Skip the deployment steps. Only connect information and tools you are authorized to use with that instance. This guide does not assume a hosted sign-up service or separate private user accounts inside an diff --git a/website/content/en/guide/collect.md b/website/content/en/guide/collect.md index 4c71dc4..fc415cd 100644 --- a/website/content/en/guide/collect.md +++ b/website/content/en/guide/collect.md @@ -1,10 +1,38 @@ --- title: Run a Collection +outline: false description: Execute one Source collection and distinguish its Source ID from its Job ID. --- # Run a Collection +Start with a [connected interface](/guide/connect). Choose Web for your own setup, or CLI / Agent +for terminal instructions. + + + + + diff --git a/website/content/en/guide/connect-cli.md b/website/content/en/guide/connect-cli.md index c849a39..9572698 100644 --- a/website/content/en/guide/connect-cli.md +++ b/website/content/en/guide/connect-cli.md @@ -8,8 +8,9 @@ description: Connect the command-line tool to an existing InKCre instance. You need a ready Core URL and the instance's private JWT secret. Obtain them from your deployment or its operator. Keep the secret private; it grants instance authority, not an isolated personal login. -The CLI handles setup operations that do not yet have a Web form. It connects over HTTPS; it does -not run another server on your computer. +The CLI is primarily for your trusted Agent, and also supports manual terminal use. For your own +interactive setup, start with [Connect to Your Instance](/guide/connect) and choose the Web app. The +CLI connects over HTTPS; it does not run another server on your computer. 1. Install [Python](https://www.python.org/downloads/) 3.12 or later if needed. Check `python --version` in a terminal; use `python3` if that is your system's command name. diff --git a/website/content/en/guide/connect.md b/website/content/en/guide/connect.md new file mode 100644 index 0000000..95fc69f --- /dev/null +++ b/website/content/en/guide/connect.md @@ -0,0 +1,52 @@ +--- +title: Connect to Your Instance +outline: false +description: Use the Web app yourself, or connect a trusted Agent through the CLI. +--- + +# Connect to Your Instance + +Use **client-web** for your own day-to-day setup and reading. The **CLI** is primarily an interface +for your trusted Agent, and also works for operators who prefer a terminal. You do not need to +configure every interface before starting. The selector on these guides changes the instructions, +not your instance, and keeps your choice while navigating the site. + + + + + + +Next: [Collect Your First Source](/guide/first-source). diff --git a/website/content/en/guide/daily-use.md b/website/content/en/guide/daily-use.md index b1fc69b..8f004f7 100644 --- a/website/content/en/guide/daily-use.md +++ b/website/content/en/guide/daily-use.md @@ -6,42 +6,20 @@ description: Connect the Web app and everyday tools, with optional AI organizati # Use Your Information Start with a working instance and [a successful search](/guide/search). Retain your Core URL, -PostgREST URL, and private JWT secret. If someone else operates the instance, ask them to register -your browser Peer rather than assuming database access. +PostgREST URL, and private JWT secret. If someone else operates the instance, ask them to authorize +your access rather than assuming you may connect a browser Peer. ## Read and explore in the Web app -The browser needs its **own** Client ID. Do not reuse the Core Peer ID: the browser excludes its own -identity when looking for another Peer to execute a capability. Settings does not yet create this -record for you. - -1. In your dedicated Neon project's **SQL Editor**, select the same default branch and `neondb` - database as deployment. For a manually hosted database, use your PostgreSQL SQL client against - the initialized InKCre database instead. Run this once to register your browser: - - ```sql - INSERT INTO inkcre.peers (id, name, config) - VALUES ( - gen_random_uuid(), - 'My web browser', - '{"extension_registry_url":"https://registry.inkcre.dev"}'::jsonb - ) - RETURNING id; - ``` - - Save the returned UUID as your browser Client ID. This adds one record without replacing Core's - record. Reuse it when reconnecting this browser. - -2. Wake Core `/readyz`, then open [Web app Settings](https://app.inkcre.dev/settings). -3. Enter **PostgreSQL REST URL** = PostgREST base URL, **JWT Secret** = your saved secret, and - **Client ID** = the new browser UUID. Choose **Save**, then reload. The Clients list should - include Core, reporting online. -4. Open **Info Base**, search for the phrase that worked in the [search guide](/guide/search), and - inspect a result. Use **View content** for supported content and the graph view to follow - relationships. The list is a search surface, not every stored Block. Some source renderers need a - compatible Web Extension; the CLI's `get_text` remains a way to read Core-resolved text. -5. Bookmark the app. Another browser/device needs its own connection setup. **Export** omits the - secret and is not an info-base backup. +First [connect the Web app](/guide/connect). Current Settings generates and registers the browser's +own Client ID; do not manually create a Peer or reuse Core's identity. + +1. Open **Info Base** and search for the phrase that worked in the [search guide](/guide/search). +2. Select a result, use **View content** where supported, and explore its relationships in the + graph. The list is a search surface, not every stored Block. Some rich renderers need a + compatible browser Extension; an Agent can use the CLI's `get_text` to read Core-resolved text. +3. Bookmark the app. A new browser/device needs its own connection. **Export** omits the secret and + is not an info-base backup. ## Use your information from a terminal or AI tool diff --git a/website/content/en/guide/extensions.md b/website/content/en/guide/extensions.md new file mode 100644 index 0000000..f9f2d04 --- /dev/null +++ b/website/content/en/guide/extensions.md @@ -0,0 +1,60 @@ +--- +title: Prepare an Extension +outline: false +description: Install a collector and enable it on the client that will run it. +--- + +# Prepare an Extension + +Start with a [connected interface](/guide/connect). Your source guide supplies an Extension name, an +exact compatible version, and its Source type. An Extension is installed once in the deployment; +enabled clients share that version and configuration. Only install code you trust with the host's +authority. Do not change a working installation merely to match an example. + + + + + + +Return to [your source's guide](/guide/sources) for its settings and first-run checkpoint. diff --git a/website/content/en/guide/first-source.md b/website/content/en/guide/first-source.md index f551136..ee6fc29 100644 --- a/website/content/en/guide/first-source.md +++ b/website/content/en/guide/first-source.md @@ -5,8 +5,9 @@ description: Choose a source and complete your first collection and search. # Collect Your First Source -Start with a working instance and a [connected CLI](/guide/connect-cli). You do not need to connect -every account at once: choose one small source with an item you will recognize. +Start with a working instance and a [connected Web app or Agent](/guide/connect). The guides default +to Web instructions, with a CLI / Agent alternative. You do not need to connect every account at +once: choose one small source with an item you will recognize. ## Choose your first source @@ -18,12 +19,14 @@ includes its own prerequisites and setup. An **Extension** supplies a collector implementation; a **Source** is one configured use of it. For example, install the RSS Extension once, then create one Source per feed. Installing it does not -automatically enable it or collect anything. Enable the collector on Core, not only in the Web app. +automatically enable it or collect anything. [Prepare the Extension](/guide/extensions) on Core, +then configure the Source in the Web app. Python-only Extension installation currently needs a +one-time operator or Agent step; it does not require doing all later setup in a terminal. ## Complete the first loop -1. Follow your chosen source's guide to enable its Extension and create the Source. Retain the - returned **Source ID**. +1. Follow your chosen source's guide to enable its Extension and create the Source. Open its details + in the Web app, or retain the returned **Source ID** when using the CLI. 2. [Run a Collection](/guide/collect) and wait for the returned **Job ID** to finish. These are different IDs: the Source persists across runs; each Job represents one run. 3. [Find What You Saved](/guide/search): maintain the lexical index and search for a known item. diff --git a/website/content/en/guide/schedules.md b/website/content/en/guide/schedules.md index 45d0dc7..d4430dc 100644 --- a/website/content/en/guide/schedules.md +++ b/website/content/en/guide/schedules.md @@ -1,10 +1,49 @@ --- title: Schedule Collection and Indexing +outline: false description: Keep sources and search indexes current with explicit schedules. --- # Schedule Collection and Indexing +Start with a [connected interface](/guide/connect). Choose Web for your own setup, or CLI / Agent +for terminal instructions. + + + + + diff --git a/website/content/en/guide/search.md b/website/content/en/guide/search.md index 161c56e..ac3e18f 100644 --- a/website/content/en/guide/search.md +++ b/website/content/en/guide/search.md @@ -1,10 +1,47 @@ --- title: Find What You Saved +outline: false description: Index collected information and retrieve it by remembered words. --- # Find What You Saved +Start with a [connected interface](/guide/connect). Choose Web for your own setup, or CLI / Agent +for terminal instructions. + + + + + diff --git a/website/content/en/guide/sources.md b/website/content/en/guide/sources.md index 85ea93f..044f9a0 100644 --- a/website/content/en/guide/sources.md +++ b/website/content/en/guide/sources.md @@ -6,9 +6,14 @@ description: Choose an independent setup guide for each information source. # Connect More Sources Use the guide for the information you want to bring into InKCre. Each page starts from a working -instance and a [connected CLI](/guide/connect-cli); you do not need to complete the RSS tutorial +instance and a [connected interface](/guide/connect); you do not need to complete the RSS tutorial first. +Choose **Web app** for your own setup or **CLI / Agent** for terminal instructions. The choice +follows you between guide pages. Shared prerequisites and source limitations apply to both. The +[Extension guide](/guide/extensions) distinguishes browser setup from Core installation and calls +out operations that still need an operator or Agent. + | What you want to collect | Setup guide | What you need | | -------------------------------------------- | ------------------------------------------------ | ---------------------------------------- | | Articles from a publication | [RSS and Atom](/guide/sources/rss) | A feed URL | diff --git a/website/content/en/guide/sources/github.md b/website/content/en/guide/sources/github.md index 197b525..6fed6ab 100644 --- a/website/content/en/guide/sources/github.md +++ b/website/content/en/guide/sources/github.md @@ -1,10 +1,41 @@ --- title: GitHub Stars and Lists +outline: false description: Connect your GitHub saves to InKCre. --- # GitHub Stars and Lists +Start with a [connected interface](/guide/connect). Choose Web for your own setup, or CLI / Agent +for terminal instructions. + + + + + diff --git a/website/content/en/guide/sources/mail.md b/website/content/en/guide/sources/mail.md index 61145ac..16db53e 100644 --- a/website/content/en/guide/sources/mail.md +++ b/website/content/en/guide/sources/mail.md @@ -1,10 +1,59 @@ --- title: Email over IMAP +outline: false description: Connect your IMAP mailbox to InKCre. --- # Email over IMAP +Start with a [connected interface](/guide/connect). Choose Web for your own setup, or CLI / Agent +for terminal instructions. + + + + + diff --git a/website/content/en/guide/sources/memos.md b/website/content/en/guide/sources/memos.md index 9bd8e7b..b377fa8 100644 --- a/website/content/en/guide/sources/memos.md +++ b/website/content/en/guide/sources/memos.md @@ -1,10 +1,47 @@ --- title: Memos-Compatible Capture +outline: false description: Write notes into InKCre from a supported Memos client. --- # Memos-Compatible Capture +Start with a [connected interface](/guide/connect). Choose Web for your own setup, or CLI / Agent +for terminal instructions. + + + + + diff --git a/website/content/en/guide/sources/rss.md b/website/content/en/guide/sources/rss.md index 2f0f5aa..1baee3d 100644 --- a/website/content/en/guide/sources/rss.md +++ b/website/content/en/guide/sources/rss.md @@ -1,10 +1,46 @@ --- title: RSS and Atom +outline: false description: Install the RSS collector and collect your first feed. --- # RSS and Atom +Start with a [connected interface](/guide/connect). Choose Web for your own setup, or CLI / Agent +for terminal instructions. + + + + + diff --git a/website/content/en/guide/sources/telegram.md b/website/content/en/guide/sources/telegram.md index 0be4b17..e8a3792 100644 --- a/website/content/en/guide/sources/telegram.md +++ b/website/content/en/guide/sources/telegram.md @@ -1,10 +1,51 @@ --- title: Send or forward messages from Telegram +outline: false description: Connect a personal Telegram capture inbox to InKCre. --- # Send or forward messages from Telegram +Start with a [connected interface](/guide/connect). Choose Web for your own setup, or CLI / Agent +for terminal instructions. + + + + + diff --git a/website/content/en/guide/sources/twitter.md b/website/content/en/guide/sources/twitter.md index 0c135fc..1063d1c 100644 --- a/website/content/en/guide/sources/twitter.md +++ b/website/content/en/guide/sources/twitter.md @@ -1,19 +1,78 @@ --- title: Twitter / X Bookmarks +outline: false description: Authorize an X account and collect its bookmarked posts. --- # Twitter / X Bookmarks This guide uses the official X API to collect your own bookmarks. Start with a -[connected CLI](/guide/connect-cli), a Core Host `0.2.x` instance with a public HTTPS URL, and an X -account containing a recent bookmark whose text you recognize. +[connected interface](/guide/connect), a Core Host `0.2.x` instance with a public HTTPS URL, and an +X account containing a recent bookmark whose text you recognize. You also need an X developer app with OAuth 2.0 user authorization and access to the required API endpoints. API access can incur charges; check your app's access and billing before starting. The [X Bookmarks documentation](https://docs.x.com/x-api/posts/bookmarks/introduction) owns provider requirements. An app-only bearer token is not a substitute for authorizing your account. + + + + + **Scope:** an ordinary run reads one page, with `result_limit` from 5 to 100 (default 40), stopping at the previously seen bookmark when present. This is not a guaranteed archive of all bookmarks, folders, replies, or unbookmarks. The current `full` option also performs a single page fetch per @@ -151,7 +214,7 @@ lose. - **Finished but no new items:** check the connected handle and bookmark a new recognizable post. To stop collection, disable its Cron first. An operator can disconnect the stored account using -authenticated `DELETE /twitter/setup/account` with the same request pattern above, and separately +authenticated `DELETE /twitter/setup/account` using the CLI / Agent request pattern, and separately revoke the app in X's account settings. Disconnecting does not delete previously collected data. The alternate `twikit` backend exists, but its account-login mechanics are not this OAuth walkthrough. diff --git a/website/content/en/self-hosted/getting-started.md b/website/content/en/self-hosted/getting-started.md index f40b8d9..f6d3d3c 100644 --- a/website/content/en/self-hosted/getting-started.md +++ b/website/content/en/self-hosted/getting-started.md @@ -8,9 +8,10 @@ description: Follow the self-hosted path from deployment to collecting and using This is the [Self-Hosted](/self-hosted/) path through InKCre: you will operate your own instance. For the application's What, Why, and How, begin with [Getting Started](/getting-started). -You will use a browser, a text editor, and a few terminal commands. No InKCre development setup or -AI model is needed for the first collection-and-search journey. Each guide below is self-contained -and can also be used with an existing compatible instance. +You will use the Web app for interactive setup. A trusted Agent or operator can handle the remaining +terminal-only operations, such as installing Core collectors and scheduling indexing. No InKCre +development setup or AI model is needed for the first collection-and-search journey. Each guide +below is self-contained and can also be used with an existing compatible instance. ## 1. Deploy an instance @@ -28,7 +29,7 @@ Core URL, a PostgREST URL, your private JWT secret, and the Core Peer ID. Follow these shared user guides in order. If you already have an instance, start here. -1. [Connect the CLI](/guide/connect-cli) and verify authenticated access. +1. [Connect to your instance](/guide/connect), using the Web app or the CLI for your Agent. 2. [Collect your first source](/guide/first-source); a public RSS or Atom feed is a simple starting point. 3. [Find what you saved](/guide/search): maintain the lexical index, search, and inspect a result. From b0a6caa77acec968e2a9b8a1c8d53784d5147f84 Mon Sep 17 00:00:00 2001 From: Lan_zhijiang Date: Sun, 20 Sep 2026 18:27:38 +0800 Subject: [PATCH 09/18] =?UTF-8?q?docs(onboarding):=20=E5=AF=B9=E9=BD=90=20?= =?UTF-8?q?Web=20=E5=AE=89=E8=A3=85=E5=92=8C=E5=8F=8C=20Host=20=E5=85=BC?= =?UTF-8?q?=E5=AE=B9=E5=8F=91=E8=A1=8C=E6=B5=81=E7=A8=8B?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - 从来源指南移除必须委托 Agent 安装的前置步骤 - 明确 Core 选择、单独启用、版本保护与 Twitter 同版本向导 --- website/content/en/guide/extensions.md | 23 +++++++++++++------- website/content/en/guide/first-source.md | 4 ++-- website/content/en/guide/sources/github.md | 2 +- website/content/en/guide/sources/mail.md | 2 +- website/content/en/guide/sources/memos.md | 5 +++-- website/content/en/guide/sources/rss.md | 3 ++- website/content/en/guide/sources/telegram.md | 2 +- website/content/en/guide/sources/twitter.md | 15 ++++++------- 8 files changed, 32 insertions(+), 24 deletions(-) diff --git a/website/content/en/guide/extensions.md b/website/content/en/guide/extensions.md index f9f2d04..014669f 100644 --- a/website/content/en/guide/extensions.md +++ b/website/content/en/guide/extensions.md @@ -16,18 +16,25 @@ authority. Do not change a working installation merely to match an example. ## Prepare the Core collector -1. Open **Extensions** and look for the name from your source guide. -2. If it is absent, ask your operator or trusted Agent to install the guide's exact **Core** release - using the CLI instructions. This is currently a one-time prerequisite: **Install New Extension** - uses the browser Host and requires a browser distribution. Selecting Core below it does not turn - that form into a Python-package installer. RSS, for example, has only a Python distribution. -3. Refresh the page. In **Control Extension on Client**, select your online **Core** client, then - turn on the Extension's switch. Enabling only **This browser** does not start a Core collector. +1. Open **Extensions**. In **Control Extension on Client**, select your online **Core** client. +2. Look for the name from your source guide. If it is absent, enter its **Extension Name** and exact + **Version** in **Install New Extension**, then select **Install Extension**. The selected Core + validates the Python release; Python-only collectors such as RSS do not need a browser package. +3. Turn on the Extension's switch while Core is still selected. Installation alone does not start + it. Enabling only **This browser** does not start a Core collector. 4. If the guide requires Extension-wide settings, open **Edit Config**, enter its configuration object, and save. Source account settings belong in **Sources**, unless the guide says otherwise. 5. Open **Sources**, create a Source, and check that its **Type** is available. A listed installation alone does not prove that the collector is running. +An offline client cannot validate an installation. If installation fails, keep your entered name and +version, read the error, and check the selected client and release compatibility before trying +again. An older Core may need an update to support installation through this Web control. Do not +switch to **This browser** as a workaround for a Python-only collector. + +For an existing installation at another version, use **Change Version** only after checking both +Hosts and disabling every client using it. Version and configuration are shared across the instance. + ## Browser Extensions and setup wizards A browser Extension adds capabilities such as content rendering or a **Setup** wizard. For a release @@ -35,7 +42,7 @@ with compatible Python **and** browser distributions, enable it on Core, then se browser** and enable it there too. Open **Setup** when that button appears. Do not install a different shared version to obtain a wizard without checking both Hosts. The -[Twitter guide](/guide/sources/twitter) explains the currently published version mismatch. +[Twitter guide](/guide/sources/twitter) walks through its paired Core and browser setup.