12 Commits (9f407d83b3dd2f18b2ffb764da71ccd992f16872)

Author	SHA1	Message	Date
Micah Snyder (micasnyd)	f79df484be	Fix Excel XLM formula parser infinite loop ... The XLM formula parser failed to account for string records that claim to be longer than the formula data. This fix skips over the invalid string records. Also fixed an unrelated XLM parsing bug where BIFF name records weren't handled on builds lacking the json-c library, resulting in verbose error output. See https://bugzilla.clamav.net/show_bug.cgi?id=12639	4 years ago
Micah Snyder (micasnyd)	b9ca6ea103	Update copyright dates for 2021 ... Also fixes up clang-format.	4 years ago
Micah Snyder	b589762814	Windows: Fix unicode filename and file share scans ... At least some unicode filenames may fail to scan in 0.102.4+ because while Windows char* strings may be UTF8, the GetFinalPathNameByHandleA function does not return UTF8 strings and instead does lossy conversion to ASCII. To fix this, we need to use GetFinalPathNameByHandleW instead and then convert from UTF16-LE to UTF8. While fixing this bug, I found and fixed a couple other serious issues with the Win32 implementation of cli_codepage_to_utf8(). If a file is on a network share, the realpath comes back with a path name that looks like "\\\\?\\UNC\\<host>\\<share>\\...". In thi scase, the "\\\\?\\UNC\\" prefix is critical or else clamscan.exe won't be able to open the file. This patch checks for the "\\\\?\\UNC" prefix and if it exists, it keeps the prefix, else it trims the "\\\\?\\" portion as before. This should fix scanning of files on network shares.	5 years ago
Micah Snyder	860764eb16	Heuristic macro detection for imp VBA extraction ... Notably the commit adds a heuristic alert when VBA is extracted using the new VBA extraction code and similarly adds "HasMacros":true to the JSON scan properties. In addition, a change was added to the cli_sanitize_filepath() function so it converts posix pathseps to Windows pathseps on Windows and also outputs a sanitized basename pointer (optional) which is used when generating a temporary filename so that using a prefix with pathseps in it won't cause file creation failures (observed with --leave-temps where original filenames are incorporated into temporarily filenames). Included soem error handling improvements for cli_vba_scandir() to better track alert and macro detections. Downgraded utf8 conversion error messages to debug messages because they are too verbose in files with invalid filenames (observed in some malware). Changed the xlm macro and vba project temp filenames to include "xlm_macros" and "vba_project" prefix, to make it easier to find them. Relocated XLM and VBA temp files from the top-level tmp directory to the current sub_tmpdir, so tempfiles for a given scan are more organized.	5 years ago
Micah Snyder	b1dbf93f0b	Fix newly introduced VBA/XLM OLE2 bugs ... Fix an infinite loop in the new XLM macro parser. Fix error handling, resource cleanup in OLE2 parser. Fix issues tracking detected "viruses" in VBA & OLE2 parsers affecting non-allmatch (regular) scan mode, wherein multiple viruses may be found but each record lost and the overall detection comes up clean. Also silence switch() fall-through warning for WORD/PPT/XL/HWP (OOXML) file type fall-throughs to the ZIP parser (because they are zips). Also silence switch() fall-through warning when handling the limits- exceeded error types, checking for the limits-exceeded heuristic, and continuing on to bail out with a clean verdict.	5 years ago
Micah Snyder (micasnyd)	244ff86cad	XLM: Fix coverity memory corruption warning ... 294429: Negative check for fd_out occurs after a call to fdopen where the value must not be negative. Coverity interprets this as a high severity issue, even though it really isn't. Removing the needless check should silence the false positive.	5 years ago
Micah Snyder (micasnyd)	6198778903	Additional XLM parser error handling fixes ... Improve error handling for functions that read the XLM BIFF temp-files. Improve resource cleanup to alleviate Coverity false positive issue.	5 years ago
Micah Snyder (micasnyd)	8081a6b06c	Fix new XLM parser stack overflow ... Fixes a stack overflow that resulted in stack corruption and general mayhem. This bugfix only applies to the 0.103 dev branch. The issue was caused by buffering formatted XLM macro content to a small buffer without regard for possible overflow. Instead of buffering manually, use of snprintf and later cli_writen were replaced with direct calls to fwrite / fprintf / fputc.	5 years ago
Andrew	035265b96f	Bug fixes related to the recent HFS+/VBA/OLE2/XLM code changes ... This commit includes bug fixes and minor modifications based on warnings generated by Coverity. These include: - 287096 - In cli_xlm_extract_macros: Leak of memory or pointers to system resources (CWE-404). This was a legitimate leak of a generated temp filename and could occur frequently. - 287095 - In scan_for_xlm_macros: Use of an uninitialized variable. The uninitialized value (state.length) was likely never used unitialized, but we now initialize it just in case. - 287094 - In cli_vba_readdir_new: Out-of-bounds access to a buffer (CWE-119). This looks like a copy-paste error and was a legitimate read past the bounds of a buffer in an error case. - 284479 - In hfsplus_walk_catalog: All paths that lead to this null pointer comparison already dereference the pointer earlier (CWE-476). In certain cases a NULL pointer could be returned in the success case of hfsplus_scanfile, which was not handled correctly. This case may have been prevented in practice by an earlier check, but adding a check for NULL just in case. - 284478 - In hfsplus_walk_catalog: A value assigned to a variable is never used. ret would be set if zlib's inflateEnd function fails. The fix is to just not set ret in this case, since the error doesn't seem fatal (although would result in a memory leak by the zlib code...). - 284477 - In hfsplus_check_attribute: Pointer is checked against null but then dereferenced anyway. I just took out the NULL check of record and recordSize, since the code requires these values to not be NULL elsewhere and there's no way an error could occur as currently used (stack var addresses are passed via these parameters). I also fixed up some of the function identifiers in debug print messages.	5 years ago
Micah Snyder	9b9999d778	Rename core scanning functions ... Many of the core scanning functions' names no longer represent their specific purpose or arguments. This commit aims to make the names more intuitive. Names are now prefixed with "magic" if they involve file-typing and file-type parsing. In addition, each function now includes the type of input being scanned whether its "desc", "fmap", or "buff". Some of the APIs also now specify "type" to indicate that a type other than "ANY" may be passed in to select the type rather than use file type magic for type recognition. | current name | new name | | ------------------------- | --------------------------------- | | magic_scandesc() | cli_magic_scan() | | cli_magic_scandesc_type() | <delete> | | cli_magic_scandesc() | cli_magic_scan_desc() | | cli_base_scandesc() | cli_magic_scan_desc_type() | | cli_partition_scandesc() | <delete> | | cli_map_scandesc() | magic_scan_nested_fmap_type() | | cli_map_scan() | cli_magic_scan_nested_fmap_type() | | cli_mem_scandesc() | cli_magic_scan_buff() | | cli_scanbuff() | cli_scan_buff() | | cli_scandesc() | cli_scan_desc() | | cli_fmap_scandesc() | cli_scan_fmap() | | cli_scanfile() | cli_magic_scan_file() | | cli_scandir() | cli_magic_scan_dir() | | cli_filetype2() | cli_determine_fmap_type() | | cli_filetype() | cli_compare_ftm_file() | | cli_partitiontype() | cli_compare_ftm_partition() | | cli_scanraw() | scanraw() |	5 years ago
Micah Snyder	005cbf5a37	Record names of extracted files ... A way is needed to record scanned file names for two purposes: 1. File names (and extensions) must be stored in the json metadata properties recorded when using the --gen-json clamscan option. Future work may use this to compare file extensions with detected file types. 2. File names are useful when interpretting tmp directory output when using the --leave-temps option. This commit enables file name retention for later use by storing file names in the fmap header structure, if a file name exists. To store the names in fmaps, an optional name argument has been added to any internal scan API's that create fmaps and every call to these APIs has been modified to pass a file name or NULL if a file name is not required. The zip and gpt parsers required some modification to record file names. The NSIS and XAR parsers fail to collect file names at all and will require future work to support file name extraction. Also: - Added recursive extraction to the tmp directory when the --leave-temps option is enabled. When not enabled, the tmp directory structure remains flat so as to prevent the likelihood of exceeding MAX_PATH. The current tmp directory is stored in the scan context. - Made the cli_scanfile() internal API non-static and added it to scanners.h so it would be accessible outside of scanners.c in order to remove code duplication within libmspack.c. - Added function comments to scanners.h and matcher.h - Converted a TDB-type macros and LSIG-type macros to enums for improved type safey. - Converted more return status variables from `int` to `cl_error_t` for improved type safety, and corrected ooxml file typing functions so they use `cli_file_t` exclusively rather than mixing types with `cl_error_t`. - Restructured the magic_scandesc() function to use goto's for error handling and removed the early_ret_from_magicscan() macro and magic_scandesc_cleanup() function. This makes the code easier to read and made it easier to add the recursive tmp directory cleanup to magic_scandesc(). - Corrected zip, egg, rar filename extraction issues. - Removed use of extra sub-directory layer for zip, egg, and rar file extraction. For Zip, this also involved changing the extracted filenames to be randomly generated rather than using the "zip.###" file name scheme.	5 years ago
Jonas Zaddach (jzaddach)	d5a733ef90	XLM (Excel 4.0) macro detection and extraction ... XLM is a macro language in Excel that was used before VBA (before 1996). It is still parsed and executed by modern Excel and is gaining popularity with malware authors. This patch adds rudimentary support for detecting and extracting Excel 4.0 (XLM) macros. The code is based on Didier Steven's plugin_biff for oletools.py.	5 years ago