Ki8dZddlZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z m Z mZddlmZddlmZddlmZejeZedz Zd efd Zd efd Zd e eeffd Zd e eefd dfdZd e fdZd e fdZej Z!ddeded efdZ"GddeZ#dS)aSingularity/Apptainer persistent container environment. Security-hardened with --containall, --no-home, capability dropping. Supports configurable resource limits and optional filesystem persistence via writable overlay directories that survive across sessions. N)Path)AnyDictOptional)get_hermes_home)BaseEnvironment)is_interruptedzsingularity_snapshots.jsonreturncxtjdrdStjdrdStd)zLocate the apptainer or singularity CLI binary. Returns the executable name (``"apptainer"`` or ``"singularity"``). Raises ``RuntimeError`` with install instructions if neither is found. apptainer singularityzNeither 'apptainer' nor 'singularity' was found in PATH. Install Apptainer (https://apptainer.org/docs/admin/main/installation.html) or Singularity and ensure the CLI is available.)shutilwhich RuntimeError;/home/ubuntu/hermes-agent/tools/environments/singularity.py_find_singularity_executablersI |K  { |M""}  :  rcpt} tj|dgddd}nB#t$rt d|dtj$rt d|dwxYw|jd kr>|jd d }t d|d |jd ||S)zPreflight check: resolve the executable and verify it responds. Returns the executable name on success. Raises ``RuntimeError`` with an actionable message on failure. versionT capture_outputtexttimeoutz:Singularity backend selected but the resolved executable 'z1' could not be executed. Check your installation.'z6 version' timed out. The runtime may be misconfigured.rNz version' failed (exit code z): ) r subprocessrunFileNotFoundErrorrTimeoutExpired returncodestderrstrip)exeresultr#s r_ensure_singularity_availabler'-s ' ( (C  )          > > > >     $    K K K K    A$$&&tt, O O O1B O Ov O O    Js +?A*ctr< tjtS#t $rYnwxYwiSN)_SNAPSHOT_STOREexistsjsonloads read_text Exceptionrrr_load_snapshotsr0Os\ :o7799:: :    D  Is*A AAdatactjddtt j|ddS)NTparentsexist_ok)indent)r*parentmkdir write_textr,dumps)r1s r_save_snapshotsr<XsG   ===tz$q999:::::rctjd}|r(t|}|dd|Sddlm}|dz }td}|rntj|tjrO|tjdd z d z }|ddt d ||S|dd|S) a Get the best directory for Singularity sandboxes. Resolution order: 1. TERMINAL_SCRATCH_DIR (explicit override) 2. TERMINAL_SANDBOX_DIR / singularity (shared sandbox root) 3. /scratch (common on HPC clusters) 4. ~/.hermes/sandboxes/singularity (fallback) TERMINAL_SCRATCH_DIRTr3r)get_sandbox_dirr z/scratchUSERhermesz hermes-agentz Using /scratch for sandboxes: %s) osgetenvrr9tools.environments.baser?r+accessW_OKloggerinfo)custom_scratch scratch_pathr?sandboxscratch user_scratchs r_get_scratch_dirrNas Y566NN++ 4$777777777o-/G:G~~BIgrw7768!    NNOQR S S SLLLLLC!!!!!!!!> ?!!!!!!!!!!sW#L>B.L-A8I'2(I''AL:L LK?-L.L?LLL L ceZdZdZ ddeded ed ed ed ed edeffd ZdZ dddddeded edzdedzde f dZ dZ xZ S)SingularityEnvironmentaHardened Singularity/Apptainer container with resource limits and persistence. Security: --containall (isolated PID/IPC/mount namespaces, no host home mount), --no-home, writable-tmpfs for scratch space. The container cannot see or modify the host filesystem outside of explicitly bound paths. Persistence: when enabled, the writable overlay directory is preserved across sessions so installed packages and files survive cleanup/restore. ~<rFdefaultrTcwdrcpumemorydiskpersistent_filesystemtask_idc $t||t|_t ||j|_dt jjdd|_ d|_ ||_ ||_ d|_ ||_||_|j rQt!dz } | dd| d|z |_ |j dd|dS) N)rrrhermes_ Fzhermes-overlaysTr3zoverlay-)super__init__r'rUrlrTuuiduuid4hex instance_id_instance_started _persistent_task_id _overlay_dir_cpu_memoryrNr9_start_instance) selfrTrrrrsrtrurvrw overlay_base __class__s rr|zSingularityEnvironment.__init__s S'222799&udo>>  %sz(Singularity: binding skills dir %s -> %sz8Singularity: could not load credential/skills mounts: %sz--memoryMz--cpusTxrzFailed to start instance: z/Singularity instance %s started (persistent=%s)zInstance start timed out)rUextendrrrbappendtools.credential_filesrrrGrHr/debugrrrTrrrr"rr#rr!)rcmdrr mount_entry skills_mountrkr&s rrz&SingularityEnvironment._start_instances G4 NK0111   + 1 + JJ S):%;%;< = = = = JJ) * * * X e e e e e e e e99;;    H[)A&f&fKP`Da&f&f&fghhh > , 01 6577L  Hk)B&h&h\RbEc&h&h&hijjj > - !12  X X X LLSUV W W W W W W W W X EE#3A'II;rWN)r stdin_datacommandrr c|jsdddS|p|j}|p|j}||\}}||||z} n||} n|} |dks|dr d|d|}d}|jd d |d |jd d |g} ddl} gtj | tj tj | r tj n tj d| rE j | j n#t $rYnwxYwfd} t#j| d} | | |z}t-r dn)#tj$rYnwxYw| dddzddS| |kr?| d||S| d| ddjdS#t $r}d|ddcYd}~Sd}~wwxYw)NzInstance not started)outputr"roz~/zcd z && z/tmpexecz--pwdz instance://bashz-crT)stdoutr#stdinrcj jD]}|dS#t$rYdSwxYwr))rrr/)line_output_chunksprocs r_drainz.SingularityEnvironment.execute.._drainTs\ $ 44&--d333344 DDs $ 22)targetdaemon)rr6rWz [Command interrupted]g?zSingularity execution error: )rrrr_prepare_commandr`rUrtimerPopenPIPESTDOUTDEVNULLrwritecloser/ threadingThreadr monotonicpollr terminatewaitr!killjoin_timeout_resultsleepr")rrrrrreffective_timeoutwork_dir exec_command sudo_stdineffective_stdinr_timerreaderdeadlinerkrrs @@rexecutezSingularityEnvironment.execute)sw% H4BGG G#3t|?$(#'#8#8#A#A j  !j&<(:5OO  #(OO(O s??h11$77?===|==LH/T-//t\+0 T N#!z/@)8Pjooj>P D  J$$_555J$$&&&& D      %VDAAAF LLNNN((+<rs"   &&&&&&&&&&,,,,,,333333******  8 $ $!/##&BBc"sDc3h;$sCx.;T;;;;$: $    !).""33S3c3C3333t'''''_'''''r