jj0dZddlmZddlZddlZddlZddlZddlmZm Z ddl m Z m Z m Z mZmZmZejdZdZdZd ZeeeehZd Zed Gd dZd[dZd\dZd]dZd^dZd_dZd`d!Zdad#Z dbd(Z!eGd)d*Z"ej#d+Z$dcd.Z%ddd1Z&ded3Z'dfd5Z( dgdhdBZ)didjdGZ*dkdIZ+eGdJdKZ,dddLdldNZ-d_dOZ.dmdQZ/ddRdndUZ0dodVZ1dpdWZ2dqdYZ3gdZZ4dS)ruProgressive tool disclosure ("tool search") for Hermes Agent. When enabled, MCP and non-core plugin tools are replaced in the model-visible tools array by three bridge tools — ``tool_search``, ``tool_describe``, ``tool_call`` — and surfaced on demand. Core Hermes tools never defer. Design constraints this module is built around (see ``openclaw-tool-search-report`` for the full rationale): * Core tools defined in ``toolsets._HERMES_CORE_TOOLS`` are *never* deferred. Always-load means always-load. No exceptions. * The threshold gate runs every assembly: when deferrable tools would consume less than ``threshold_pct`` of the model's context window (default 10%), tool search is a no-op and the tools array passes through unchanged. * The catalog is stateless across turns and tools-array assemblies. It is rebuilt from the current tool-defs list every time. This is the lesson from OpenClaw's cron regression (openclaw/openclaw#84141): a session-keyed catalog that drifts out of sync with the live tool registry produces silent tool dropouts. * Bridge tools route through ``model_tools.handle_function_call`` exactly like a direct call, so guardrails, plugin pre/post hooks, approval flows, and tool-result truncation all fire identically. * Display and trajectory unwrap is implemented here so the user (CLI activity feed, gateway, saved trajectories) always sees the underlying tool, not the bridge. ) annotationsN) dataclassfield)AnyDictIterableListOptionalTupleztools.tool_search tool_search tool_describe tool_callg@T)frozencTeZdZUdZded<ded<ded<ded<edd ZdS)ToolSearchConfigzDResolved, validated tool-search configuration for a single assembly.strenabledfloat threshold_pctintsearch_default_limitmax_search_limitrawrreturn'ToolSearchConfig'c |dur|ddddS|dur|ddddSt|ts|ddddSt|d d}|d vrd }n|d vrd}n |d vr|}nd}t |dd}tdtd|}tdtdt|dd}tdt|t|dd}|||||S)ayBuild a config from a raw dict / bool / None. Accepts the legacy bool shape (``tools.tool_search: true``) and the dict shape (``tools.tool_search: {enabled: auto, ...}``). Validates and clamps every numeric field; unknown values fall back to safe defaults rather than raising, so a typo in user config does not break the agent. Tautog$@)rrrrFoffr)true1yeson)false0no)rr$r rY@2rr) isinstancedictrgetstriplower _safe_floatmaxmin _safe_int)clsr enabled_rawrrrrs 6/home/ubuntu/.hermes/hermes-agent/tools/tool_search.pyfrom_rawzToolSearchConfig.from_rawHs $;;3vT,-DDD D %<<3uD,-DDD D#t$$ D3vT,-DDD D#'')V4455;;==CCEE . . .GG 0 0 0GG 1 1 1!GGG#CGGO$<$V-AE-IJKK  0 00r9cheZdZUdZded<ded<ded<ded<ded<ee Zd ed <d S) CatalogEntryzEOne deferrable tool, in a form the bridge tools can search and serve.rr\ descriptionDict[str, Any]schemasource source_name)default_factory List[str]_tokensN)r:r;r<r=r>rlistrr@r9r7rr snOO IIIKKKt444G444444r9rz [A-Za-z0-9]+textrcR|sgSdt|DS)Nc6g|]}|Sr@)r0).0ts r7 z_tokenize..s 7 7 7!AGGII 7 7 7r9) _TOKEN_REfindall)rs r7 _tokenizers1  7 7y0066 7 7 77r9rsrc|dpi}|dd}|ddpd}|dpidpi}d|}|ddd dd dd d}|d|d|S) uXBuild the search-text blob for a deferrable tool. Includes the tool name (with underscores broken into words so BM25 can match against query terms), the description, and the names of the top-level parameters. Schema bodies are deliberately excluded — indexing them adds noise without improving recall in our measurement. rnr\ror parameters properties _.-ry)r.joinkeysreplace)rsrtr\descparams param_names name_wordss r7_entry_search_textr!s    !rB 66&"  D 66- $ $ *Dvvl##)r..|<<BF((6;;==))Kc3''//S99AA#sKKSSTWY\]]J / /4 / /+ / //r9Tuple[str, str]c ddlm}||}|dS|jdr d|jfSd|jfS#t $rYdSwxYw)z=Return (source_kind, source_name) for a registered tool name.rr_N)otherroramcpplugin)rcr`rdrerfrNrgs r7_classify_sourcer3s ++++++""4(( = = = # #F + + *5=) )%-(( }}sA "A A AAList[CatalogEntry]c Pg}|D]}|dpi}|dd}|s2|ddpd}t|\}}t|||||tt |}|||S)zBuild the deferred-tool catalog from a tool-defs list. Caller is expected to pass only the deferrable subset (``classify_tools`` returns it as the second element). rnr\ror)r\rrrrr)r.rrrrrp) rjcatalogrsrtr\rrrrhs r7 build_catalogrAs #%G VVJ   %2vvfb!!  vvmR((.B.t44 #04455     u Nr9?? query_tokens doc_tokens doc_lengths List[int]avg_dldoc_freqDict[str, int]n_docsk1bc |sdSd}t|} i} |D]} | | ddz| | <|D]} || d} | dkrtjd|| z dz| dzz z}| | d}|dkr_||dzz||d|z || zt |dz zzzz }|||zz }|S)uStandard BM25 score for one query against one document. Inlined small implementation rather than adding a dependency. Performance is fine — the catalog is bounded by N (tools) typically < 500, and we score against the in-memory tokens list. r(rr*g?g?)r|r.rlogr2)rrrrrrrrscoredldoc_tfrqdfidftfnorms r7 _bm25_scorer[s s E ZBF ))JJq!$$q(q    \\!Q   77 hqFRK#-"s(;;<< ZZ1   77 R!V}R1q51r6C.s 333a3qy>>333r9r*g?c|dS)Nrr@)xs r7z search_catalog..s adr9T)keyreversecg|]\}}|Sr@r@)rrrTs r7rz"search_catalog..s ) ) )$!QA ) ) )r9N) rsumr2r|setrr.rrpr0r\sort)rrrrrrrrTseenrrscoredrhsqls r7search_catalogrzs eqjj U##L  437333K   C $4$4a 8 8 8F!H 1119~~ 1 1A",,q!,,q0HQKK 1 \\F/1F&&  em[& & * * q55 MM1e* % % % , [[]] , ,EUZ%%'''' sEl+++ KKNNDK111 ) )&%. ) ) ))r9deferred_countc  d|dtdtd}dtdtd}dtd }d t|d d d ddddddgdddd t|d dd ddidgdddd t|d d ddd dddddgdddgS)uBuild the bridge tool schemas to inject in place of deferred tools. The schemas are intentionally short — every byte added here is a byte the user pays on every turn. Descriptions are tuned to be unambiguous about the call sequence the model should follow. zSearch zu additional tools that are loaded on demand. Returns up to ``limit`` matches with name and description. Follow with `z0` to load a tool's full parameter schema, then `zs` to invoke it. Tools listed at the top of this system prompt are already available and do not need to be searched.z4Load the full JSON schema for one tool returned by `z`. Required before `z'` if the tool's parameters are unknown.zhInvoke a deferred tool by name with the given arguments. Argument shape matches the tool's schema (see `zM`). Policy, hooks, and approvals run exactly as for any directly-listed tool.rnobjectstringzIKeywords describing the capability you need (e.g. 'create github issue').)typerintegerz/Maximum number of results to return. Default 5.)rrr)rrrequiredr\rr)rrnr\z-Exact tool name (as returned by tool_search).zExact tool name to invoke.z,Arguments for the tool, matching its schema.)r\ argumentsr)TOOL_DESCRIBE_NAMETOOL_CALL_NAMETOOL_SEARCH_NAME)r desc_search desc_describe desc_calls r7bridge_tool_schemasrs N. N N# N N  N N N T?O T T* T T T  E+= E E E(*$%-+v"" %.+\"" # #")     ,*,$$,+Z!!# "(      $&($%-+G!! %-+Y&& # #"( 5    O< <r9cReZdZUdZded<ded<dZded<dZded <dZded <d S) AssemblyResultzrrrr@r9r7rrsaFF####OOONOr9r)rrOptional[ToolSearchConfig]c f|t}d|D}t|\}}|st|dSt|}t |||s;t|dt ||t |pd|jdz zStt |}||z}t |pd|jdz z} t dt |t ||| t|d t ||| S) a#Return the tool-defs list the model should actually see. When tool search is inactive (off, no deferrable tools, or below threshold), this is a passthrough. When active, MCP and plugin tools are stripped from the visible list and replaced with the three bridge tools. Core tools are *never* deferred regardless of config. Idempotent: calling with bridge tools already in the input is a no-op (they classify as non-core/non-deferrable but their names are reserved, so they are filtered out of the deferrable set). Nctg|]5}|dpidtv3|6S)rnr\)r.rb)rrss r7rz&assemble_tool_defs..'sOTTTrFF:&&,"11&99ARRRRRRr9F)rjrrr))rjrrrrzZtool_search activated: %d core/visible tools kept, %d deferred (~%d tokens, threshold ~%d)T) rKrurrrr|rrrrOinfo) rjrrincomingrqrrrbridgeresultrs r7assemble_tool_defsrsk"~TTYTTTH)22GZ CEBBBB4Z@@ 6#4n E E z??- ."5A&:NQV:V!WXX     !Z 1 1F v FN/aF4H54PQRR KKd G c*oo'8:J :))    r9c|tvSrD)rb)r\s r7is_bridge_toolrOs $ $$r9rhcL|j|j|j|jpddddS)Nroir\rrrr)rhs r7_format_search_hitrSs5 ,()/R#6   r9)rargscurrent_tool_defsc 4|t}t|dpd}|st jddidS|d}||j}n6td t|j t||j}t|\}}t|}t||| } t j|t|d | Dd dS) z?Execute the ``tool_search`` bridge tool. Returns a JSON string.Nrroerrorzquery is requiredFrzrr*)rc,g|]}t|Sr@)r)rhs r7rz(dispatch_tool_search..ts!888a&q))888r9)rtotal_availablematches)rKrr.r/r}r~rr2r3rr4rurrr|) rrrr raw_limitrrrrrhitss r7dispatch_tool_searchr ]s" ~ !!'R ( ( . . 0 0E Nz7$78uMMMM!!I+As62IiId4e4effgg"#455MAzJ''G '5 6 6 6D :w<<884888    r9c Ft|dpd}|stjddidSt |stjdd|didSt |\}}|D]v}|d pi}|d|krDtj||d d|d id dcSwtjdd|d idS)zAExecute the ``tool_describe`` bridge tool. Returns a JSON string.r\rorzname is requiredFr'z' is not a deferrable tool. If you see it in the tools list already, call it directly; otherwise check the spelling against tool_search.rnrrrz<' is not currently available. Re-run tool_search to refresh.)rr.r/r}r~riru)rrr\rrrrsrts r7dispatch_tool_describerxs txx%2 & & , , . .D Mz7$67eLLLL "4 ( (z _D___    ##455MAz## VVJ   %2 66&>>T ! !:!vvmR88 ff\266" ### # # # " :WTWWW   r9ct}|D]S}|dpidd}|r$t|r||Tt |S)aeReturn the set of deferrable tool names present in ``tool_defs``. ``tool_defs`` is expected to be the *pre-assembly* tool list for the current session's toolset scope (i.e. what ``get_tool_definitions(skip_tool_search_assembly=True)`` returns for the session's enabled/disabled toolsets). The resulting set is the universe of tools the session may legitimately reach through ``tool_call``. Used as a scoping gate by both the ``model_tools`` bridge dispatch and the ``tool_executor`` unwrap so a restricted-toolset session can never invoke an out-of-scope tool via the bridge. rnr\ro)rr.riaddrZ)rjnamesrsr\s r7scoped_deferrable_namesrsxeeEz""(b--fb99  +D11  IIdOOO U  r93Tuple[Optional[str], Dict[str, Any], Optional[str]]ct|dpd}|sdidfS|tvr did|dfS|d}|i}t |tr: t j|}n$#t j$r}did|fcYd}~Sd}~wwxYwt |tsdid fSt|s did |d fS||dfS) a?Parse a ``tool_call`` invocation into (underlying_name, args, error_msg). Used by: * the dispatcher in ``model_tools.handle_function_call``, * the display layer (so the activity feed shows the underlying tool), * the trajectory recorder. On parse error, returns ``(None, {}, error_message)``. r\roNz$tool_call requires a 'name' argumentztool_call cannot invoke 'z' (it is itself a bridge tool)rz)tool_call 'arguments' is not valid JSON: z'tool_call 'arguments' must be an objectrz|' is not a deferrable tool. If it appears in the model-facing tools list already, call it directly instead of via tool_call.) rr.r/rbr,r}loadsJSONDecodeErrorr-ri)rr\raw_argsrTs r7resolve_underlying_callrsd txx%2 & & , , . .D @R???    RYTYYYYYxx $$H(C  M Mz(++HH# M M MLLLL L L L L L L M h % %CRBBB "4 ( ( R G G G G   4 s?BB5#B0*B50B5)rrrrbrrrrKrirurrrrrrrr rrr)rArrBrrr)rArrBrrr)rr)rrU)r\rrr])rjrkrrl)rjrvrr)rrrrrrrr])rrrr)rsrrr)r\rrr)rjrkrr)rr)rrrrrrrrrrrrrrrrrr)r)rrrrrrrr)rrrrk)rjrkrrrrrr)rhrrr)rrrrkrrrr)rrrrkrr)rjrkrrU)rrrr)5r= __future__rr}loggingrre dataclassesrrtypingrrrr r r getLoggerrOrrrrZrbrrr4r1rKr[rirurrrcompilerrrrrrrrrrrrr rrr__all__r@r9r7r#s6#"""""  ((((((((================  . / / !$I/1C^TUU $2 2 2 2 2 2 2 2 j / / / /&    489999"1111@  5 5 5 5 5 5 5  5 BJ ' ' 8888 0000$    :-1>(*(*(*(*(*`TTTTx  %))- 666666|%%%%?C68(    B   r9