j\LdZddlZddlZddlZejeZehdZehdZ ej dd dvZ gdZ d Zejd ed Zd Zejd edejZejdejZejdZejdZejdejZejdZejdZejdZejdZejdejZejdZejdde zdzZdddddd d!e d"e!d#e!d$e!d%e d&e d'e fd(Z"d)e d'e fd*Z#d+e d'e fd,Z$d-e d'e fd.Z%d-e d'e fd/Z&d-e d'e fd0Z'd-e d'e fd1Z(d2d2d3d-e d4e)d5e)d'e fd6Z*d7e d'e fd8Z+e,d9e DZ-d-e d'e)fd:Z.d;Z/d-e d'e)fd<Z0Gd=d>ej1Z2dS)?a2Regex-based secret redaction for logs and tool output. Applies pattern matching to mask API keys, tokens, and credentials before they reach log files, verbose output, or gateway logs. Short tokens (< 18 chars) are fully masked. Longer tokens preserve the first 6 and last 4 characters for debuggability. N>x-amz-signaturejwtkeyauthcodetokenapikeysecretapi_keysessionid_tokenpassword signature access_token client_secret refresh_token>rrrrr r r r r private_keyr authorizationrrHERMES_REDACT_SECRETStrue>1onyesr)$zsk-[A-Za-z0-9_-]{10,}zghp_[A-Za-z0-9]{10,}zgithub_pat_[A-Za-z0-9_]{10,}zgho_[A-Za-z0-9]{10,}zghu_[A-Za-z0-9]{10,}zghs_[A-Za-z0-9]{10,}zghr_[A-Za-z0-9]{10,}zxox[baprs]-[A-Za-z0-9-]{10,}zAIza[A-Za-z0-9_-]{30,}zpplx-[A-Za-z0-9]{10,}zfal_[A-Za-z0-9_-]{10,}zfc-[A-Za-z0-9]{10,}zbb_live_[A-Za-z0-9_-]{10,}zgAAAA[A-Za-z0-9_=-]{20,}zAKIA[A-Z0-9]{16}zsk_live_[A-Za-z0-9]{10,}zsk_test_[A-Za-z0-9]{10,}zrk_live_[A-Za-z0-9]{10,}zSG\.[A-Za-z0-9_-]{10,}zhf_[A-Za-z0-9]{10,}zr8_[A-Za-z0-9]{10,}znpm_[A-Za-z0-9]{10,}zpypi-[A-Za-z0-9_-]{10,}zdop_v1_[A-Za-z0-9]{10,}zdoo_v1_[A-Za-z0-9]{10,}zam_[A-Za-z0-9_-]{10,}zsk_[A-Za-z0-9_]{10,}ztvly-[A-Za-z0-9]{10,}zexa_[A-Za-z0-9]{10,}zgsk_[A-Za-z0-9]{10,}zsyt_[A-Za-z0-9]{10,}zretaindb_[A-Za-z0-9]{10,}zhsk-[A-Za-z0-9]{10,}zmem0_[A-Za-z0-9]{10,}zbrv_[A-Za-z0-9]{10,}zxai-[A-Za-z0-9]{30,}z9(?:API_?KEY|TOKEN|SECRET|PASSWORD|PASSWD|CREDENTIAL|AUTH)z([A-Z0-9_]{0,50}z&[A-Z0-9_]{0,50})\s*=\s*(['\"]?)(\S+)\2z(?:api_?[Kk]ey|token|secret|password|access_token|refresh_token|auth_token|bearer|secret_value|raw_secret|secret_input|key_material)z("z")\s*:\s*"([^"]+)"z!(Authorization:\s*Bearer\s+)(\S+)z#(bot)?(\d{8,}):([-A-Za-z0-9_]{30,})zH-----BEGIN[A-Z ]*PRIVATE KEY-----[\s\S]*?-----END[A-Z ]*PRIVATE KEY-----zK((?:postgres(?:ql)?|mysql|mongodb(?:\+srv)?|redis|amqp)://[^:]+:)([^@]+)(@)z2eyJ[A-Za-z0-9_-]{10,}(?:\.[A-Za-z0-9_=-]{4,}){0,2}z (\+[1-9]\d{6,14})(?![A-Za-z0-9])z;(https?|wss?|ftp)://([^\s/?#]+)([^\s?#]*)\?([^\s#]+)(#\S*)?z+(https?|wss?|ftp)://([^/\s:@]+):([^/\s@]+)@z`\b((?:GET|POST|PUT|PATCH|DELETE|HEAD|OPTIONS|TRACE|CONNECT)\s+[^ \t\r\n\"']*?)\?([^ \t\r\n\"']+)zH^[A-Za-z_][A-Za-z0-9_.-]*=[^&\s]*(?:&[A-Za-z_][A-Za-z0-9_.-]*=[^&\s]*)+$z(?>> mask_secret("sk-proj-abcdef1234567890") 'sk-p...7890' >>> mask_secret("short") # fully masked '***' >>> mask_secret("") # empty default '' >>> mask_secret("", empty="(not set)") # empty override '(not set)' >>> mask_secret("long-token", head=6, tail=4, floor=18) '***' Nz...)len)r$rr r!r"r#s 1/home/ubuntu/.hermes/hermes-agent/agent/redact.py mask_secretr)sOP   5zzEETEl . .udUVV} . ..rc0|sdSt|dddS)uOMask a log token — conservative 18-char floor, preserves 6 prefix / 4 suffix.rr)rr r!)r))rs r( _mask_tokenr.s( u u11B 7 7 77r*queryc`|s|Sg}|dD]~}d|vr|||d\}}}|tvr||di||d|S)aRedact sensitive parameter values in a URL query string. Handles `k=v&k=v` format. Sensitive keys (case-insensitive) have values replaced with `***`. Non-sensitive keys pass through unchanged. Empty or malformed pairs are preserved as-is. &=z=***)splitappend partitionlower_SENSITIVE_QUERY_PARAMSjoin)r/partspairr_r$s r(_redact_query_stringr<s  E C   d?? LL    s++ Q 99;;1 1 1 LLC & & & & LL     88E??r*textcfdtjdtfd}t||S)uScan text for URLs with query strings and redact sensitive params. Catches opaque tokens that don't match vendor prefix regexes, e.g. `https://example.com/cb?code=ABC123&state=xyz` → `...?code=***&state=xyz`. mr%c|d}|d}|d}t|d}|dpd}|d||d||S) Nrr://?groupr<)r?scheme authoritypathr/fragments r(_subz&_redact_url_query_params.._subsGGAJJ wwqzz$QWWQZZ00771::#@@Y@@@u@h@@@r*)reMatchstr_URL_WITH_QUERY_REsubr=rMs r(_redact_url_query_paramsrTsD AASAAAA  ! !$ - --r*c:td|S)zStrip `user:password@` from HTTP/WS/FTP URLs. DB protocols (postgres, mysql, mongodb, redis, amqp) are handled separately by `_DB_CONNSTR_RE`. c^|dd|ddS)NrArErBz:***@rHr?s r(z&_redact_url_userinfo..(s+QWWQZZ55AGGAJJ555r*)_URL_USERINFO_RErRr=s r(_redact_url_userinfor\!s%   55   r*cfdtjdtfd}t||S)zARedact sensitive query params in HTTP access-log request targets.r?r%c~|d}t|d}|d|S)NrArBrFrG)r?prefixr/s r(rMz6_redact_http_request_target_query_params.._sub/s<$QWWQZZ00""5"""r*)rNrOrP_HTTP_REQUEST_TARGET_QUERY_RErRrSs r((_redact_http_request_target_query_paramsra-s>##S#### ) , ,T4 8 88r*c|rd|vsd|vr|St|s|St|S)uNRedact sensitive values in a form-urlencoded body. Only applies when the entire input looks like a pure form body (k=v&k=v with no newlines, no other text). Single-line non-form text passes through unchanged. This is a conservative pass — the `_redact_url_query_params` function handles embedded query strings.  r1) _FORM_BODY_REmatchstripr<r[s r(_redact_form_bodyrg6sZ 44<<3d??   tzz|| , ,  - --r*F)force code_filerhric"|dSt|tst|}|s|S|s ts|St|rtd|}|sHd|vrd}t ||}d|vr"d|vrd}t||}d|vsd |vrtd |}d|vrd }t||}d |vrd |vrtd|}d|vrtd|}d|vrtd|}d|vrd|vrt|}d|vrd}t||}|S)uApply all redaction patterns to a block of text. Safe to call on any string -- non-matching text passes through unchanged. Enabled by default. Disable via security.redact_secrets: false in config.yaml. Set force=True for safety boundaries that must never return raw secrets regardless of the user's global logging redaction preference. Set code_file=True to skip the ENV-assignment and JSON-field regex patterns when the text is known to be source code (e.g. MAX_TOKENS=*** constants, "apiKey": "test" fixtures). Prefix patterns, auth headers, private keys, DB connstrings, JWTs, and URL secrets are still redacted. Performance: each regex pattern is gated behind a cheap substring pre-check (e.g. ``"=" in text`` for ENV assignments, ``"://" in text`` for URLs, ``"eyJ" in text`` for JWTs). On a typical hermes log line (no secrets) this drops the 13-pattern scan from ~5.6us to ~1.8us per record (-68%). The pre-checks are conservative — false positives still run the full regex, which then doesn't match. False negatives are impossible because every regex requires the gated substring to match. NcFt|dS)NrAr.rHrXs r(rYz'redact_sensitive_text..gs AGGAJJ(?(?r*r2c|d|d|d}}}|d|t||S)NrArBrCr2rHr.)r?namequoter$s r( _redact_envz*redact_sensitive_text.._redact_envlsS%&WWQZZQWWQZZUeCCC E(:(:CECCCr*:"c|d|d}}|dt|dS)NrArBz: "rsrn)r?rr$s r( _redact_jsonz+redact_sensitive_text.._redact_jsonss>WWQZZU77+e"4"47777r* uthorization UTHORIZATIONcr|dt|dzS)NrArBrnrXs r(rYz'redact_sensitive_text..}s'aggajj;qwwqzz#:#::r*ch|dpd}|d}||dS)NrArrBz:***rW)r?r_digitss r(_redact_telegramz/redact_sensitive_text.._redact_telegrams9WWQZZ%2FWWQZZF*f*** *r*BEGINz-----z[REDACTED PRIVATE KEY]rEc\|dd|dS)NrArrCrWrXs r(rYz'redact_sensitive_text..s(qwwqzz,J,Jaggajj,J,Jr*eyJcFt|dS)NrrlrXs r(rYz'redact_sensitive_text..s[%<%<r*r1+c|d}t|dkr|dddz|ddzS|dddz|ddzS)NrArBz****r)rHr')r?phones r( _redact_phonez,redact_sensitive_text.._redact_phones`GGAJJE5zzQRaRy6)E"##J66!9v%bcc 2 2r*) isinstancerP_REDACT_ENABLED_has_known_prefix_substring _PREFIX_RErR_ENV_ASSIGN_RE_JSON_FIELD_RE_AUTH_HEADER_RE _TELEGRAM_RE_PRIVATE_KEY_RE_DB_CONNSTR_RE_JWT_RErg_SIGNAL_PHONE_RE)r=rhrirqrur{rs r(redact_sensitive_textrFs, |t dC 4yy  _ #4((G~~??FF  : $;; D D D"%%k488D $;;3$;; 8 8 8"%%lD99D 4!7!7"" : :     d{{ + + + 0$77$7d??""#;TBB }}!!"J"JDQQ }}{{<rs<#$Ar*cDtfdtDS)zReturn True if ``text`` contains any known credential prefix substring. Used as a cheap pre-check before invoking the expensive ``_PREFIX_RE``. c3 K|]}|vV dSr)rrr=s r(rz._has_known_prefix_substring..s'55QqDy555555r*)any_PREFIX_SUBSTRINGSr[s`r(rrs) 5555"4555 5 55r*) zGET zPOST zPUT zPATCH zDELETE zHEAD zOPTIONS zTRACE zCONNECT cl|tfdtDS)z?Cheap pre-check before scanning for access-log request targets.c3 K|]}|vV dSrr)rmethoduppers r(rz-_has_http_method_substring..s'EE6vEEEEEEr*)rr_HTTP_METHOD_SUBSTRINGS)r=rs @r(_has_http_method_substringrs4 JJLLE EEEE-DEEE E EEr*cBeZdZdZdfd Zdejdeffd ZxZ S) RedactingFormatterz9Log formatter that redacts secrets from all log messages.N%c @tj|||fi|dSr)super__init__)selffmtdatefmtstylekwargs __class__s r(rzRedactingFormatter.__init__s,gu7777777r*recordr%cdt|}t|Sr)rformatr)rroriginalrs r(rzRedactingFormatter.formats&77>>&))$X...r*)NNr) __name__ __module__ __qualname____doc__rlogging LogRecordrPr __classcell__)rs@r(rrsrCC888888/W./3//////////r*r)3rrosrN getLoggerrlogger frozensetr7_SENSITIVE_BODY_KEYSgetenvr6r_PREFIX_PATTERNS_SECRET_ENV_NAMEScompiler_JSON_KEY_NAMES IGNORECASErrrrrrrrQrZr`rdr8rrPintr)r.r<rTr\rargboolrrtuplerrrr Formatterrrr*r(rs  8 $ $ $)%%%,!y"""4")3V<<BBDDHbb%%%PQU+UUU Z-/---M "*(Mrz* "*O RM "*%  2:ABB  RZ2:2!+ M!! O RZSXX&6777:OO ,/,/,/ ,/ ,/  ,/  ,/  ,/ ,/ ,/,/,/,/^8s8s8888, .3 .3 . . . . s s    93939999 .C .C . . . . 7