all2md.utils.security

Security utilities for all2md conversion modules.

This module provides security validation functions to prevent unauthorized access to local files and malicious ZIP archive processing.

Functions

  • resolve_file_url_to_path: Resolve file:// URL to canonical filesystem path

  • validate_local_file_access: Check if access to a local file path is allowed

  • validate_zip_archive: Pre-validate ZIP archives for security threats

  • sanitize_language_identifier: Sanitize code fence language identifiers

all2md.utils.security.sanitize_null_bytes(content: str) → str

Remove null bytes and zero-width characters that can bypass XSS filters.

This function removes various null-like and zero-width Unicode characters that attackers may use to bypass XSS sanitization filters. These characters can be used to hide malicious payloads or break parser assumptions.

Removed characters: - \x00 (NULL byte) - \ufeff (BOM/Zero Width No-Break Space) - \u200b (Zero Width Space) - \u200c (Zero Width Non-Joiner) - \u200d (Zero Width Joiner) - \u2060 (Word Joiner)

Parameters:

content (str) – Content to sanitize

Returns:

Sanitized content with dangerous characters removed

Return type:

str

Examples

>>> sanitize_null_bytes("Hello\\x00World")
'HelloWorld'
>>> sanitize_null_bytes("Test\\u200bZero\\u200cWidth")
'TestZeroWidth'
>>> sanitize_null_bytes("Normal text")
'Normal text'

Notes

This function is used by HTML and other parsers to prevent XSS attacks that rely on null bytes or zero-width characters to bypass security filters.

See also

HtmlToAstConverter.convert_to_ast

Uses this function to sanitize HTML input

all2md.utils.security.resolve_file_url_to_path(file_url: str) → Path

Resolve a file:// URL to a canonical filesystem path.

This function provides consistent path resolution for file:// URLs across the codebase. It handles various file URL formats including relative paths, Windows paths, and UNC paths. All paths are resolved to absolute canonical form to prevent path traversal attacks.

Supported file:// URL formats: - Unix/Linux absolute: file:///path/to/file - Windows drive letters: file:///C:/path/to/file - Windows UNC paths: file://server/share/file - Relative paths: file://./file or file://../file - Relative to CWD: file://filename

Parameters:

file_url (str) – The file:// URL to resolve

Returns:

Resolved absolute path object

Return type:

Path

Raises:

ValueError – If file_url is not a file:// URL

Examples

>>> resolve_file_url_to_path("file:///etc/passwd")
Path('/etc/passwd')
>>> resolve_file_url_to_path("file://./image.png")
Path('/current/working/directory/image.png')
>>> resolve_file_url_to_path("file:///C:/Users/file.txt")  # Windows
Path('C:/Users/file.txt')

Notes

This function is used by both validate_local_file_access() and HTML parser to ensure consistent path resolution and prevent TOCTOU vulnerabilities.

Always call validate_local_file_access() before using the resolved path to access files, to ensure security policies are enforced.

all2md.utils.security.validate_local_file_access(file_url: str, allow_local_files: bool = False, local_file_allowlist: list[str] | None = None, local_file_denylist: list[str] | None = None, allow_cwd_files: bool = True) → bool

Validate if access to a local file URL is allowed based on security settings.

Supports various file URL formats including: - Unix/Linux: file:///path/to/file - Windows drive letters: file:///C:/path/to/file - Windows UNC paths: file://server/share/file - Relative paths: file://./file or file://../file

Parameters:

  • file_url (str) – The file:// URL to validate

  • allow_local_files (bool, default False) – Master switch for local file access. If False, no local files are allowed.

  • local_file_allowlist (list[str] | None, default None) – List of directories allowed for local file access (when allow_local_files=True)

  • local_file_denylist (list[str] | None, default None) – List of directories denied for local file access

  • allow_cwd_files (bool, default True) – Allow local files from current working directory and subdirectories. Only applies when allow_local_files=True.

Returns:

True if access is allowed, False otherwise

Return type:

bool

Examples

>>> validate_local_file_access("file:///etc/passwd", allow_local_files=False)
False
>>> validate_local_file_access("file://./image.png", allow_local_files=False)
False
>>> validate_local_file_access("file://./image.png", allow_local_files=True, allow_cwd_files=True)
True
>>> validate_local_file_access("file:///C:/Users/file.txt", allow_local_files=True)  # Windows
True
all2md.utils.security.validate_zip_archive(file_path: str | Path, max_compression_ratio: float = 100.0, max_uncompressed_size: int = 1073741824, max_entries: int = 10000) → None

Validate a ZIP archive for security threats before processing.

This function performs pre-validation checks on ZIP archives to detect potential security threats like zip bombs, path traversal attacks, and excessive resource consumption.

Parameters:

  • file_path (str or Path) – Path to the ZIP archive to validate

  • max_compression_ratio (float, default 100.0) – Maximum allowed compression ratio (uncompressed/compressed)

  • max_uncompressed_size (int, default 1073741824) – Maximum total uncompressed size in bytes (default: 1GB)

  • max_entries (int, default 10000) – Maximum number of entries in the archive

Raises:

Examples

>>> validate_zip_archive("document.docx")
# Passes if the file is a safe ZIP archive

>>> validate_zip_archive("malicious.zip")
ZipFileSecurityError: ZIP archive has suspicious compression ratio: 1000:1
all2md.utils.security.validate_tar_archive(file_path: str | Path, max_compression_ratio: float = 100.0, max_uncompressed_size: int = 1073741824, max_entries: int = 10000) → None

Validate a TAR archive for security threats before processing.

This function performs pre-validation checks on TAR archives to detect potential security threats like tar bombs, path traversal attacks, and excessive resource consumption.

Parameters:

  • file_path (str or Path) – Path to the TAR archive to validate

  • max_compression_ratio (float, default 100.0) – Maximum allowed compression ratio (uncompressed/compressed)

  • max_uncompressed_size (int, default 1073741824) – Maximum total uncompressed size in bytes (default: 1GB)

  • max_entries (int, default 10000) – Maximum number of entries in the archive

Raises:
all2md.utils.security.validate_7z_archive(file_path: str | Path, max_compression_ratio: float = 100.0, max_uncompressed_size: int = 1073741824, max_entries: int = 10000) → None

Validate a 7Z archive for security threats before processing.

This function performs pre-validation checks on 7Z archives to detect potential security threats like archive bombs, path traversal attacks, and excessive resource consumption.

Parameters:

  • file_path (str or Path) – Path to the 7Z archive to validate

  • max_compression_ratio (float, default 100.0) – Maximum allowed compression ratio (uncompressed/compressed)

  • max_uncompressed_size (int, default 1073741824) – Maximum total uncompressed size in bytes (default: 1GB)

  • max_entries (int, default 10000) – Maximum number of entries in the archive

Raises:
all2md.utils.security.validate_rar_archive(file_path: str | Path, max_compression_ratio: float = 100.0, max_uncompressed_size: int = 1073741824, max_entries: int = 10000) → None

Validate a RAR archive for security threats before processing.

This function performs pre-validation checks on RAR archives to detect potential security threats like archive bombs, path traversal attacks, and excessive resource consumption.

Parameters:

  • file_path (str or Path) – Path to the RAR archive to validate

  • max_compression_ratio (float, default 100.0) – Maximum allowed compression ratio (uncompressed/compressed)

  • max_uncompressed_size (int, default 1073741824) – Maximum total uncompressed size in bytes (default: 1GB)

  • max_entries (int, default 10000) – Maximum number of entries in the archive

Raises:
all2md.utils.security.validate_safe_extraction_path(output_dir: str | Path, zip_entry_name: str) → Path

Validate and return a safe extraction path for a ZIP/archive entry to prevent path traversal.

This function prevents Zip Slip attacks by ensuring that extracted files cannot escape the intended output directory through absolute paths or parent directory traversal (..). Works for ZIP and other archive formats.

Parameters:

  • output_dir (str or Path) – The base directory where files should be extracted

  • zip_entry_name (str) – The filename from the archive entry (ZipInfo.filename or archive member name)

Returns:

A safe, validated absolute path for extraction

Return type:

Path

Raises:

ArchiveSecurityError – If the path contains dangerous patterns or would escape output_dir

Examples

>>> validate_safe_extraction_path("/tmp/out", "subdir/file.txt")
Path('/tmp/out/subdir/file.txt')

>>> validate_safe_extraction_path("/tmp/out", "../etc/passwd")
ArchiveSecurityError: Unsafe path in archive entry: ../etc/passwd

>>> validate_safe_extraction_path("/tmp/out", "/etc/passwd")
ArchiveSecurityError: Unsafe path in archive entry: /etc/passwd

Notes

This function is critical for preventing Zip Slip vulnerabilities (CVE-2018-1000117 and related). Always use this when extracting archive entries to the filesystem.

See also

validate_zip_archive

Pre-validate ZIP archives for security threats

validate_tar_archive

Pre-validate TAR archives for security threats

all2md.utils.security.sanitize_language_identifier(language: str) → str

Sanitize code fence language identifier to prevent markdown injection.

Code fence language identifiers must only contain safe characters to prevent markdown injection via malicious language strings. This function validates and sanitizes language strings by checking against a safe pattern of alphanumeric characters, underscores, hyphens, and plus signs.

Parameters:

language (str) – Raw language identifier string to sanitize

Returns:

Sanitized language identifier, or empty string if invalid

Return type:

str

Examples

>>> sanitize_language_identifier("python")
'python'
>>> sanitize_language_identifier("c++")
'c++'
>>> sanitize_language_identifier("python\\nmalicious")
''
>>> sanitize_language_identifier("python javascript")
''
>>> sanitize_language_identifier("x" * 100)
''

Notes

This function is used by both HTML and Markdown parsers to ensure consistent security validation of code block language identifiers.

all2md.utils.security.validate_safe_output_directory(output_dir: str | Path, allowed_base_dirs: list[str | Path] | None = None, block_sensitive_paths: bool = True) → Path

Validate that an output directory is safe for file operations.

This function prevents path traversal attacks by detecting and blocking relative paths that escape the current working directory (e.g., ../../etc/). Absolute paths are allowed but can be validated against an allowlist.

Security Model

By default, this function: - BLOCKS: Relative paths that traverse outside CWD (e.g., ../../../etc/) - BLOCKS: Paths to sensitive system directories (optional, via block_sensitive_paths) - ALLOWS: Relative paths within CWD (e.g., ./attachments, subdir/files) - ALLOWS: Absolute paths (explicit intent, but check sensitive paths)

param output_dir:

The output directory to validate

type output_dir:

str or Path

param allowed_base_dirs:

Optional allowlist of base directories. If provided, output_dir must be within one of these directories. When None, uses the default security model described above.

type allowed_base_dirs:

list[str | Path] | None, default None

param block_sensitive_paths:

Block paths to common sensitive system directories like /etc, /sys, /proc. Only applies when allowed_base_dirs is None.

type block_sensitive_paths:

bool, default True

returns:

The validated, resolved absolute path

rtype:

Path

raises SecurityError:

If the path contains path traversal patterns or targets sensitive locations

Examples

>>> validate_safe_output_directory("./attachments")
Path('/current/working/directory/attachments')

>>> validate_safe_output_directory("../../../etc/")
SecurityError: Path traversal detected

>>> validate_safe_output_directory("/tmp/safe-output")
Path('/tmp/safe-output')

>>> validate_safe_output_directory("/tmp/out", allowed_base_dirs=["/tmp"])
Path('/tmp/out')

Notes

This function focuses on preventing PATH TRAVERSAL attacks (the actual vulnerability identified in the security review) rather than restricting all paths outside CWD, which would break legitimate use cases.

all2md.utils.security.validate_user_regex_pattern(pattern: str) → None

Validate user-supplied regex pattern to prevent ReDoS attacks.

This function checks user-supplied regex patterns for dangerous constructs that could lead to catastrophic backtracking (Regular Expression Denial of Service - ReDoS attacks). Patterns with nested quantifiers or other backtracking-prone structures are rejected.

Parameters:

pattern (str) – The regex pattern to validate

Raises:

SecurityError – If the pattern is too long or contains dangerous constructs

Examples

>>> validate_user_regex_pattern(r"^/docs/")
# Returns None (safe pattern)

>>> validate_user_regex_pattern(r"(a+)+")
SecurityError: Regex pattern contains dangerous nested quantifiers

>>> validate_user_regex_pattern("x" * 1000)
SecurityError: Regex pattern exceeds maximum length

Notes

This function is conservative and may reject some complex but safe patterns. This is intentional to ensure security. Common safe patterns include: - Simple anchors: ^, $ - Character classes: [a-z], [0-9] - Single-level quantifiers: a+, b*, c{2,5} - Alternations without quantifiers: (cat|dog) - Simple groups: (abc)+

Dangerous patterns that are rejected include: - Nested quantifiers: (a+)+, (b*)* - Quantified groups with inner quantifiers: (a+){2,} - Lookaheads/lookbehinds with quantifiers: (?=.*)+, (?!test)* - Lookaheads/lookbehinds containing quantifiers: (?=.*a) - Overlapping alternations: (a|ab)*, (foo|foobar)+ - Multiple nested groups: ((a+) - Greedy wildcards with quantifiers: .*+, .+*

For more information on ReDoS attacks, see the OWASP documentation on Regular Expression Denial of Service attacks.

all2md.utils.security.is_relative_url(url: str) → bool

Check if a URL is a relative URL.

Relative URLs do not have a scheme and typically start with #, /, ./, ../, or ?.

Parameters:

url (str) – URL to check

Returns:

True if URL is relative, False otherwise

Return type:

bool

Examples

>>> is_relative_url("#section")
True
>>> is_relative_url("/path/to/file")
True
>>> is_relative_url("./file.html")
True
>>> is_relative_url("../parent/file.html")
True
>>> is_relative_url("?query=param")
True
>>> is_relative_url("https://example.com")
False
>>> is_relative_url("javascript:alert(1)")
False
all2md.utils.security.is_url_scheme_dangerous(url: str) → bool

Check if a URL uses a dangerous scheme.

Dangerous schemes include javascript:, vbscript:, data:text/html, and others that can be used for XSS attacks or malicious code execution.

Parameters:

url (str) – URL to check

Returns:

True if URL uses a dangerous scheme, False otherwise

Return type:

bool

Examples

>>> is_url_scheme_dangerous("https://example.com")
False
>>> is_url_scheme_dangerous("javascript:alert('xss')")
True
>>> is_url_scheme_dangerous("data:text/html,<script>alert('xss')</script>")
True
>>> is_url_scheme_dangerous("/relative/path")
False
>>> is_url_scheme_dangerous("vbscript:msgbox('xss')")
True
all2md.utils.security.validate_url_scheme_safe(url: str, context: str = 'URL') → None

Validate that a URL does not use a dangerous scheme.

This function raises a ValueError if the URL uses a dangerous scheme, making it suitable for strict validation contexts.

Parameters:

  • url (str) – URL to validate

  • context (str, default "URL") – Context description for error messages (e.g., “Link”, “Image”)

Raises:

ValueError – If URL uses a dangerous scheme

Examples

>>> validate_url_scheme_safe("https://example.com")
# No exception raised

>>> validate_url_scheme_safe("javascript:alert(1)")
ValueError: URL uses dangerous scheme

>>> validate_url_scheme_safe("/relative/path")
# No exception raised