12 Commits (b30d9c54b242aee062ae52d93e6714714d591f8e)

Author	SHA1	Message	Date
Micah Snyder	ed57b85074	CMake, LLVM, Win32: Fix link issue when LLVM lib list are full paths	3 years ago
Micah Snyder	8bf70207d5	CMake: Fix LLVM linking issues: libclamav_rust, -ltinfo ... We must pass the LLVM library dependencies to the libclamav_rust build.rs script so it links the libclamav_rust unit test executable with LLVM. Also: - We can remove the libtinfo dependency that was hardcoded for the LLVM 3.6 support, and must remove it for the build to work on Alpine, macOS. - Also, increased the libcheck default timeout from 60s to 300s after experiencing a failure while testing this. - Also made one of the valgrind suppressions more generic to account for inline optimization differences observed in testing this.	3 years ago
Micah Snyder	fd587c741c	Image fuzzy hash: new logical sub-signature feature ... Add a new logical signature subsignature type for matching on images with image fuzzy hashes. Image fuzzy hash subsigantures follow this format: fuzzy_img#<hash>#<dist> In this initial implementation, the hamming distance (dist) is ignored and only exact fuzzy hash matches will alert. Fuzzy hash matching is only performed for supported image types. Also: removed some excessive debug log messages on start-up. Fixed an issue where the signature name (virname) is being allocated and stored for every subsignature or even ever sub-pattern in an AC-pattern (i.e. NDB sig or LDB subsig) containing a `{n-m}` or `*` wildcard. This fix is only for LDB subsigs though. NDB signatures are still allocaing one virname per sub-pattern. This fix was required because I needed a place to store the virname with fuzzy-hash subsignatures. Storing it in the fuzzy-hash subsig metadatathe way AC-pattern, PCRE, and BComp subsigs were doing it wouldn't work because it would cross the C-Rust FFI boundary and giving pointers to Rust allocated stuff is dicey. Not to mention native Rust strings are different thatn C strings. Anyways, the correct thing to do was to store the virname with the actual logical signature. TODO: Keep track of NDB signatures in the same way and store the virname for NDB sigs there instead of in AC-patterns so that we can get rid of the virname field in the AC-pattern struct.	3 years ago
Scott Hutton	39a9c3b956	Improve cdiff_apply performance ... * Work with data as &[u8] instead String/&str to avoid unnecessary UTF-8 validation and reuse read buffers. * Make error handling more concise * Address Clippy-raised issues	3 years ago
micasnyd	527ce73449	Rust bindgen: Disable layout tests ... Bindgen makes tests for struct layout by default. We don't want these tests because pointer width won't be the same everywhere.	3 years ago
micasnyd	4d34bc10b8	Rust bindgen: Only generate if CMake MAINTAINER_MODE is ON ... In testing, I found that libclang.so/clang.dll is required by bindgen and was not found on all of our test machines. To resolve this we will only generate sys.rs bindings when CMake MAINTAINER_MODE option is "ON". This is unfortunate that we have to commit generated source to version control. But as a benefit it makes rust-analyzer happier when editing a workspace that hasn't yet been compiled. And it makes it more reasonable that the generated sys.rs file generated to the source directory and not the build directory (something we hadn't resolved yet).	3 years ago
micasnyd	ba0a21c31f	Generate Rust-bindings for some internal C-headers ... Use bindgen to generate Rust-bindings for some libclamav internal functions and structures in an new "sys.rs" module. "sys.rs" will be generated at build-time and unfortunately must be dropped in the source/libclamav_rust/src directory rather than under the build directory. As far as I know, Cargo/Rust provide no way to set an include path to the build directory to separately place generated files. TODO: Verify if this is true and move sys.rs to the build directory if possible. Using the new bindings with: - the logging module. - the cdiff module. Also: - Removed clamav_rust.h from .gitignore, because we generate it in the build directory now. - Removed the hand-written bindings from the cbindgen exclusions. lib.rs has an annotation that prevents cbindgen from looking at sys.rs. - Fixed a `u8` -> `c_char` type issue in cdiff in the cli_getdsig() call parameters.	3 years ago
Scott Hutton	0872cc2053	cdiff: Rust-ify and error handling clean-up. ... Rustify cdiff_apply() and clean up error handling: - Restore [some] safety and clean up error handling. - Use rust-crypto sha2 instead of OpenSSL's. Fix signedness of cli_versig2() dsig parameter. c_char may be an i8 or u8 depending on platform: https://doc.rust-lang.org/src/std/os/raw/mod.rs.html#91-133 Rustify cmd_close(): - Consolidate DEL/XCHG records. - Tidy up ADD handling. - Various error handling cleanup, etc.	3 years ago
micasnyd	5698e3285c	CMake-Rust: Only execute cbindgen for build, not tests ... Generate c-bindings header in build (binary) directory, not source directory (if different).	3 years ago
Micah Snyder	b362747705	Rust: Link libclamav test executable with libstdc++ on linux ... This is needed for static builds with static dependencies.	3 years ago
Scott Hutton	278ba2923e	Integrate cbindgen via build.rs	4 years ago
Scott Hutton	37e5b573aa	Add Rust logging module unit test & integrate with CMake / CTest ... Add a basic unit test for the new libclamav_rust `logging.rs` module. This test simply initializes logging and then prints out a message with each of the `log` macros. Also set the Rust edition to 2018 because the default is the 2015 edition in which using external crates is very clunky. For the Rust test support in CMake this commit adds support for cross-compiling the Rust tests. Rust tests must be built for the same LLVM triple (target platform) as the rest of the project. In particular this is needed to build both x64 and x86 packages on a 64bit Windows host. For Alpine, we observed that the LLVM triple for the host platform tools may be either: - x86_64-unknown-linux-musl, or - x86_64-alpine-linux-musl To support it either way, we look up the host triple with `rustc -vV` and use that if the musl libc exists. This is a big hacky and unfortunately means that we probably can't cross-compile to other platforms when running on a musl libc host. There are probably improvements to be made to improve cross compiling support. The Rust test programs must link with libclamav, libclammspack, and possibly libclamunrar_iface and libclamunrar plus all of the library dependencies for those libraries. To do this, we pass the path of each library in environment variables when building the libclamav_rust unit test program. Within `libclamav_rust/build.rs`, we read those environment variables. If set, we parse each into library path and name components to use as directives for how to build the unit test program. See: https://doc.rust-lang.org/cargo/reference/build-scripts.html Our `build.rs` file ignores the library path environment variables if thye're not set, which is necessary when building the libclamav_rust library and when libclamunrar isn't static and for when not linking with a libiconv external to libc. Rust test programs are built and executed in subdirectory under: <target>/<llvm triple>/<config>/deps where "target" for libclamav_rust tests is set to <build>/unit_tests For example: clamav/build/unit_tests/x86_64-pc-windows-msvc/debug/deps/clamav_rust-7e1343f8a2bff1cc.exe Since this program isn't co-located with the rest of the libraries we also have to set environment variables so the test program can find and load the shared libraries: - Windows: PATH - macOS: DYLD_LIBRARY_PATH We already set LD_LIBRARY_PATH when not Windows for similar reasons. Note: In build.rs, we iterate references to LIB_ENV_LINK & Co because older Rust versions do implement Iterator for [&str].	4 years ago