http://open-source-security-software.net/project/moloch/releases.atom 2024-11-18T16:19:29.384529+00:00 python-feedgen moloch v2.4.0 2023-11-07T14:49:01+00:00 ## Changelog * NOTE - RHEL/Centos 6 is no longer supported, Node 12 required * NOTE - New encoding of packetPos, set gapPacketPos=false for old encoding * NOTE - 2.4.x will be the last versions to support ES 6 * release - node 12.18.2, glib 2.64.5, curl 7.72.0 * release - Ubuntu 20 support * viewer - aes256Encryption now defaults to true * viewer - added a clear cache button to ES Admin tab * viewer - quote expressions with [ or ] in them * viewer - add button to only show data nodes on ES Nodes tab * viewer - files tab can now show the packet pos encoding * viewer - ES Indices tab can now show the avg doc size per index * viewer - ES Nodes tab can now show shards and segments per node * capture - http2 decoding for PRI * h2 sessions * capture - set http2 protocol when alpn is h2 * capture - upgrade h2c http2 decoding * capture - no longer use internal libpcap function * capture - simple writer supports maxFileTimeM (PR #1506) * capture - new packetPos encoding saves 10%-20% overall ES space * capture - remove old disk writer methods, use simple or simple-nodirect now * wise - simple UI * wise - support json file format config files 2023-11-07T14:49:01+00:00 moloch v2.4.1 2023-11-07T14:49:24+00:00 ## Changelog * NOTICE - db.pl upgrade is required * NOTICE - the elasticsearch and usersElasticsearch variables must start with http:// or https:// * release - node 12.18.4 * viewer - fixed export pcap from actions menu not working * viewer - capture stats/graph now uses regex instead of wildcard * viewer - support -reindex indices * viewer - log more info when can't open a file * viewer - lastpass boxes removed * viewer - can now edit ILM values from ES Admin tab if ./db.pl ilm has been used previously * viewer - handle hunts with bad regex better * viewer - change capture stats default length to 200 * viewer - fix password change with aes256Encryption turned on * viewer - handle hunts when nodes are down better * wise - UI improvements * wise - theatstream mode sqlite3 no longer copies the db, use sqlite3-copy for old behaviour * parliament - show bits instead of bytes * db - new reindex command * capture - http2 header fields were not always indexed correctly * capture - fix g_hash_table_contains warning * capture - rules can use special ip values ipv4 and ipv6 now * moloch_update_geo.sh - fix possible security issue 2023-11-07T14:49:24+00:00 moloch v2.4.2 2023-11-07T14:49:47+00:00 ## Changelog * NOTICE - db.pl upgrade is required * NOTICE - this is the last version to support ES 6 * release - node 12.19.0 * viewer - support utf8 chars in content-disposition * viewer - add capture process restart to timeline graphs * viewer - add "bookmarks" apply a view's expression to the search input without issuing a query * viewer - fix anonymous users settings not being saved * viewer - share hunts between users * viewer - move all common client bundling, scripts, and npm modules to top level * viewer - display business hours on sessions timeline graph * viewer - fix multi mpls header decoding * viewer - fix viewer crashing when pcap file not available * viewer - new getSessionBySearch setting * viewer - decode vxlan packets better * viewer - add help icon * viewer - added startTime and runningTime capture stats * capture - QUIC version 5x detection * capture - smtp decoding handles clients that break utf8 section incorrectly * capture - fix a json parsing fail would cause next json parse to fail even if good * capture - support 0x6558 Ether Bridging * wise - threatstream improvements when using the sqlite db * db - fix rm-node to delete over 10k items, and bad count display * tests - use our oui/rir files 2023-11-07T14:49:47+00:00 moloch v2.7.0 2023-11-07T14:50:08+00:00 ## Changelog * NOTICE - Requires ES 7.4 or newer * NOTICE - Moloch to Arkime rebranding in UI, everything else still Moloch * all - ES 7 updates, fix most depreciated warnings (mappings/templates still remain) * viewer - fix mpls decoding * viewer - new themes and logo selection * capture - fix http CONNECT response parsing * capture - New pcap-over-ip reader support * db - can import gz files directly now * db - fix issues with version importing 2023-11-07T14:50:08+00:00 moloch v2.7.1 2023-11-07T14:50:28+00:00 ## Changelog * release - glib 2.66.2, curl 7.73.0, nghttp2 1.42.0 * wise - fix UI queries hanging * viewer - fix anonymous user settings not saving * viewer - fix lastUsed time not always saving to ES * capture - new packet dedup feature https://arkime.com/settings#packet-deduplication-settings * capture - close pfring on exit (issue #1538) * capture - fix http2 parsing crash * db - Moloch to Arkime text fixes 2023-11-07T14:50:28+00:00 moloch v3.0.0 2023-11-07T14:50:49+00:00 ## Changelog * BREAKING - Elasticsearch - ES 7.10.0 or later required * BREAKING - if not using a ES prefix, the prefix arkime_ will now be used * BREAKING - multies - multiESNodes requires a name: and prefix: attribute per entry * BREAKING - wise - custom sources will need to be modified * BREAKING - wise - redis urls have a new standard format * BREAKING - wise - for json data keyColumn has been renamed keyPath * BREAKING - wise - now lower case lotermfield and upper case uptermfield fields * BREAKING - capture - override-ips will no longer have leading 0s for ASN * release - curl 7.78.0, node 14.17.5, glib 2.68.3, yara 4.0.2, lua 5.3.6, maxmind 1.4.3, nghttp2 1.44.0 * capture - search for GeoIP files first in /var/lib/GeoIP * capture - fixed possible ABR with time parsing * capture - fixed memory leak with dns bad hostname parsing * capture - new classifier for nfs and several rpc protocols * capture - handle larger oracle connect msgs better * capture - parse small http basic auth headers correctly * capture - rule matches can now be logged * capture - new localPcapIndex setting to enable pcap index on capture node instead of ES * capture - write long open tcp sessions every tcpSaveTimeout even if no syn * capture - fixed possible memory corruption with --flush option * capture - check max packet length in more places * capture - parse proxy-authorization header (PR #1651) * db.pl - new urlinfile://filepath where elasticsearch url is the first line of filepath file * db.pl - fix history mapping issue * viewer - add POST version session query APIs to support long expressions old GET versions still work * viewer - reorganize and standardize APIs * viewer - put npm scripts and common npm packages at top level * viewer - allow viewing of large encrypted pcap files (issue #1555) * viewer - can find nodes for pcap based on --host * viewer - in multiviewer mode can now select which clusters to search (PR #1325) * viewer - fix addTag/removeTag with multiviewer (PR #1556) * viewer - can modify some sessions2 template from esadmin tab * viewer - can modify entire shortcuts now * viewer - spigraph visualization improvements * viewer - Can set watermark settings again * viewer - Start moloch -> arkime cookie changes * viewer - dark theme improvements and new themes * viewer - can now toggle on/off capture start times * viewer - improve toolbar for packet display * viewer - standardize on . ipv6 and : ipv4 port separator * viewer - send shallower queries to ES when possible * viewer - fix cron queries when using autoGenerateId * viewer - improve session settings across sessions * viewer - fix multiple requests when changing views and start/stop times * viewer - fix setting user permissions not applying until page reload * viewer - improve column resizing * viewer - add button to open sessions cron query tagged * viewer - rename cron queries to periodic queries, fix bugs, and add data (created time, creator userId, last run time, enabled/disabled time) * viewer - add create periodic query to action menu dropdown * viewer - notifier links to results * viewer - notifier displays more data (creator userId, created time, last updated time) * viewer - cancel hunts & remove hunt name and id from sessions * viewer - shortcuts sync across all clusters if usersElasticsearch and cronQueries is set * viewer - monitor cert and key files and reload them if they change * viewer - display http request method in history page * viewer - add bad status codes for failing to fetch pcap * viewer - fix uncompress packets by requesting all packets * wise - new config UI, enable with --webconfig * wise - more moloch->arkime changes * wise - for json data can now set arrayPath for start of where to parse * wise - new nlasticsearchfile type * wise - threatstream results now cached * wise - support memcached for cache * wise - new urlinfile://filepath where config is the first line of filepath file * wise - help page * wise - valueactions can be stored in flat file, redis or elasticsearch * all - renamed rightclicks to valueactions * all - many typos fixed * all - support Elasticsearch API Keys (elasticsearchAPIKey setting) * esproxy - first version, see https://arkime.com/esproxy 2023-11-07T14:50:49+00:00 moloch v3.1.0 2023-11-07T14:51:25+00:00 ## Changelog * all - support float field type * all - support Q-in-Q ether type * all - --insecure has consistent messaging now * all - Finally added elasticsearchBasicAuth/usersElasticsearchBasicAuth can be user:pass or base64(user:pass) * capture - s3 writer uses IMDSv2 (thanks fj604) * capture - dscp src/dst (issue #1626) * capture - refactor how udp tunnels are added, just normal parsers now * capture - initial GENEVE, VXLAN-GPE support * capture - initial IETF QUIC support * capture - fix interfaceOps crash (issue #1763) * release - much improved arkime_config_interfaces.sh (thanks arkaØm) * release - node 14.18.0 * viewer - stop upload menu showing up when empty command * viewer - improve first load time by spliting bundle and lazy load items * viewer - combine multiple calls from UI into one call * viewer - added pcap expire debugging * viewer - uploadCommand can now use INSECURE-ORIGINALNAME to access uploaded filename * viewer - shortcut display improvements * viewer - shortcuts can be in arrays * viewer - shortcut wildcard support (issue #1554) * viewer - didn't decrypt pcap for large packets correctly (issue #1756) * viewer - support wiseURL for wise.js viewer plugin (issue #1758) * viewer - display "0" values * viewer - Test alert includes who requested it * viewer - fixed regex and floats not always working in array expressions * viewer - fix always putting cursor at end of input when selecting typeahead result * viewer - add typeahead results in search expression for values in a list * multies - support elasticsearchApiKey * multies - support scrolling and large queries/pagination * wise - handle empty config file on startup * wise - support ./ with value-actions * wise - support usersElasticsearchAPIKey * esproxy - elasticsearchAPIKey and elasticsearchBasicAuth support 2023-11-07T14:51:25+00:00 moloch v3.1.1 2023-11-07T14:52:01+00:00 ## Changelog * release - node 14.18.1 * addUser.js - fix not exiting on complete when certs defined * all - deal with usersElasticsearch being an array better * capture - increase max of pcapWriteSize, tpacketv3NumThreads * capture - decrease max of maxESConns, maxESRequests * viewer - fix vlan display (broke in 3.0.0) * viewer - Issue Query on Page Load "No" option changed to issue query if there is a search expression * viewer - fix position of value actions dropdown in spigraph table * wise - fix error msg about redundant parameters * wise - new mergeQuery setting for splunk 2023-11-07T14:52:01+00:00 moloch v3.2.0 2023-11-07T14:52:39+00:00 ## Changelog * release - node 14.18.2 * release - remove daily.sh, setup a cron directly now * all - refactor some shared code into common directory * capture - fix memory leak with ip4 frags and packet q overflowing * capture - standardize on config error process exiting * capture - ietf quic improvements * tests - add some auth tests to test suite * viewer - jquery upgrade * viewer - help fields display improvements * viewer - support https urls in wise plugin (issue #1777) * viewer - fix history links with && not working * viewer - userAuthCreateTmpl improvements * viewer - fix cron and database bounding queries * viewer - fix settings page not loading on pre 3.x config * viewer - cyberchef didn't always load the packets 2023-11-07T14:52:39+00:00 moloch v3.2.1 2023-11-07T14:53:17+00:00 ## Changelog * esproxy - handle sessions2 without prefix * capture - new parseHTTPHeaderValueMaxLen replaces hard coded 1024 * viewer - some hunt fixes * viewer - switch from ES bool MUST to FILTER * viewer - increase elasticsearchScrollTimeout default * viewer - new AND arrays with ][ syntax vs OR arrays with [] * viewer - fix --insecure which broke in 3.2.0 * viewer - ES Nodes has new uptime stat * viewer - fix 3.x sending to remote cluster * viewer - disable periodic queries on multiviewer * viewer - history can always toggle open and show api * wise - fixed views that used require: not working * db.pl - sync-files, add-missing, and other fixes since 3.x 2023-11-07T14:53:17+00:00 moloch v3.3.0 2023-11-07T14:53:56+00:00 ## Changelog * BREAKING - non standard pcap files now use the .arkime extension * BREAKING - for wise multiES entries, prefix: now defaults to arkime_ * BREAKING - for wise threatstream source you must create md5 index manually * release - node 14.18.3 * viewer - default to hunt reassembled packets * viewer - add descriptions to hunts * viewer - hide graph/map (speeds up large queries) * viewer - history logs es query/indices * viewer - open up to 50 sessions at a time button * viewer - make sure hunt progress bar shows up * viewer - handle corrupt pcap files better * viewer - handle hunt errors better * viewer - scrubbing won't crash on unsupported files * capture - make sure file/seq es requests have higher priority * capture - support pcap flies with 0 timestamp * capture - use .arkime file extension for non standard pcap files * capture - new _dropBySession rule op * capture - fix infite recursion - thanks albntomat0_1 * capture - improve udp/tcp header length checking * capture - improve error messages for field setting issues * capture - cache when getting pcap data from S3 (thanks pjsg) * capture - New ecsEventProvider setting * wise - switch from node-sqlite3 to better-sqlite3 package * all - support creating gzip files, set simpleGzipBlockSize * all - support creating pcap files with short packet headers, set simpleShortHeader 2023-11-07T14:53:56+00:00 moloch v3.3.1 2023-11-07T14:54:34+00:00 ## Changelog * viewer - fix displaying large packets or xored packets not always working * capture - refactor curl code based on recommendations * capture - only allow 50 packets per ip4 frag * capture - new modbus parser (thanks mcgillowen) * tests - reduce race conditions 2023-11-07T14:54:34+00:00 moloch v3.4.0 2023-11-07T14:55:12+00:00 ## Changelog * release - node 16.14.0, libpcap 1.10.1 * release - Configure script deals with / in password better * BREAKING - in header auth mode userAuthIps allows only localhost by default * wise - fix issues with redis source * wise - threatstream in sqlite3 mode opens in readonly now * wise - support -o section.var=value command line option * wise - improve json parsing to handle non arrays when expecting an array * wise - didn't always encoding number fields correctly * db.pl - added a repair command that will fix some common issues * viewer - reading packets from S3 failed * viewer - increase speed when searching match fields * viewer - fixed lastUsed when in digest auth * viewer/wise - new userAuthIps setting that has which ips auth requests can come from. header mode - default localhost, other - default all ips * viewer - record which node is cron node and warn if not found * viewer - allow floating point numbers for disk watermarks * capture - switch from deflate to gzip posting to ES, lower min gzip size to 860 * capture - ietf quic improvements * esproxy - can now create a [tee] section that will duplicate all ES calls, but ignore results * tests - Add --elasticsearch option which is actually used correctly 2023-11-07T14:55:12+00:00 moloch v3.4.1 2023-11-07T14:55:56+00:00 ## Changelog * release - node 16.14.1 * capture - new snmp parser of a few fields * capture - rules can have numeric ranges * db.pl - stop using history type name * esproxy - added queries/_mapping to GET allow list * viewer - Packets/s, Sessions/s, Dropped/s weren't accurate (thanks mcgillowen) 2023-11-07T14:55:56+00:00 moloch v3.4.2 2023-11-07T14:56:33+00:00 ## Changelog * release - node 16.14.2 * viewer - Packets/s, Sessions/s, Dropped/s didn't have correct total/average * viewer - host = $shortcut should work now * capture - support longer node names (thanks mcgillowen) * capture - host.smb.tokens wasn't defined correctly * wise - alienvault no longer uses key * tagger - support --insecure option * tests - support --insecure with tests 2023-11-07T14:56:33+00:00 moloch v4.0.0 2023-11-07T14:57:10+00:00 ## Changelog * BREAKING - Must be 3.3.0+ to upgrade to 4.x * BREAKING - systemd files auto installed, still need to enable * BREAKING - Move to roles for some permission checking, userAdmin role required to edit users * BREAKING - the version file lives in common directory now * BREAKING - new defaults maxFileSizeG=12, compressES=true * BREAKING - pcap compression is turned on by default, disable with simpleCompression=none * BREAKING - simpleGzipBlockSize renamed simpleCompressionBlockSize * BREAKING - right-click changed to value-actions in config * BREAKING - the userId search in history for admin nolonger adds the surrounding wildcards automatically * BREAKING - views & notifiers are now their own indices * release - cyberchef 9.46.5, node 16.16.0 * release - systemd files are delivered with /opt/arkime path instead of setting at install time * release - CICD tests with OpenSearch * all - Support ES 8 & OpenSearch * all - check for missing users index or no users on startup * all - update code/docs to mention OpenSearch * addUser.js - new --roles option, --admin creates superAdmin user * capture - New ecsEventDataset setting * capture - save sessions not saving packets for across restarts * capture - afpacket rewrite, improve performance & less out of order packets * capture - fix quic crash * capture - make creating fields from config/parsers/wise/tagger use ES bulk call * capture/viewer - new outer fields replace gre fields (PR #1889) * capture/viewer - initial SLL2 support (issue #2002) * capture/viewer - zstd pcap compression * chad - new plugin * cont3xt - new Cont3xt application, see https://arkime.com/cont3xt * cont3xt/viewer - share new user UI * db.pl - fix sync-files/add-missing trying to add non pcap files * db.pl - init/wipe clean up aliases that became indices * db.pl - determine data node using roles array (issue #2006) * db.pl - fix warning: Smartmatch is experimental * db.pl - ilm didn't work if no sessions2 indices * common - fix userAuthIps setting * esproxy - check gzip traffic * esproxy - improved bulk and url sanitization * parliament - added --insecure option * parliament/wise - can be configured to use shared user DB * suricata - support char 127 in json better * viewer - fixed auth fallback to digest from header mode * viewer - field-actions to display configurable menu items on field labels * viewer - share shortcuts with specific users or roles ("arkimeUser" role = old shortcut sharing) * viewer - share views with specific users or roles ("arkimeUser" role = old shortcut sharing) * viewer - share notifiers with specific users or roles (all previous notifiers will be shared with the "arkimeUser" role) * viewer - share queries with specific users or roles * viewer - share hunts with roles * viewer - rework some settings UI, try and make UX similiar when adding things * viewer - no more _moloch_shared user * viewer - configure auto-hiding map/graph on large queries using turnOffGraphDays (default = 30 days) * wise - fix some stats sorting * wise - fix UI saving of INI formatted config * wise - can configure field-actions 2023-11-07T14:57:10+00:00 moloch v4.0.1 2023-11-07T14:57:55+00:00 ## Changelog * addUser.js - remove WARNING adding first user * addUser.js - --webauthonly now sets header auth flag * all - better console output sanitization * capture - offline pcap allows more outstanding packets based on maxPacketsInQueue * db.pl - Fixed some OpenSearch compatibility * db.pl - Fixed upgrading to 4.x with no _moloch_shared user * viewer - Fix cert notbefore/notafter showing bad dates in sessions table 2023-11-07T14:57:55+00:00 moloch v4.0.2 2023-11-07T14:58:41+00:00 ## Changelog * release - cyberchef 9.48.0 * all - better console output sanitization * capture/viewer - Add TLS Certificate Organisational Unit field parsing (PR #2038) * capture - use arkime_update_geo.sh in error msg * capture - log error and exit if fields loading fails * release - Stop Configure from destroying systemd files 2023-11-07T14:58:41+00:00 moloch v4.0.3 2023-11-07T14:59:32+00:00 ## Changelog * release - cyberchef 9.54.0 * release - copy systemd files instead of soft linking * releaes - capture/viewer systemd files now After OpenSearch/Elasticsearch * capture - on short runs, field definitions weren't getting updated * capture - s3 writer sets s3Compress to false with s3WriteGzip true * capture - JA3s value was sometimes incorrect * cont3xt - fixed digest mode fetching settings from config file * db.pl - fixed init not working with OpenSearch sometimes * db.pl - will now count data or data_hot node roles * viewer - fixed showing more than 10 roles 2023-11-07T14:59:32+00:00 moloch v4.1.0 2023-11-07T15:00:18+00:00 ## Changelog * release - glib 2.72.4 cyberchef 9.55.0 flot 4.2.3 d3 7.7 * db.pl - backup/restore wasn't dealing with templates correctly * db.pl - upgrade failed if there was no moloch_shared user * db.pl - repair now fixes missing history/ecs templates * db.pl - fix users-export/users-import * cont3xt - support missing auth and userTmpl settings * cont3xt - Hide link group when no links match filter * cont3xt - Added landing page * capture - allow wise field dst.ip:port * capture - add VNI field * capture - initial tzsp reader support * capture - y2038 fixes * capture - Integer ops in rules now support a leading min or max which only sets the value if less than or greater than current value * wise - added usersElasticsearchBasicAuth setting and lmdb cache support * wise - add passivetotal value action if at least key is defined * viewer - fix es node stats for different node.roles * viewer/cont3xt - can now search roles * viewer/cont3xt - don't show change password menu item if web auth is enabled for user and disableUserPasswordUI is true 2023-11-07T15:00:18+00:00 moloch v4.2.0 2023-11-07T15:01:04+00:00 ## Changelog * release - node 16.19.1, support node v18 * release - fix arch build issues * release - EL9 build uses sha256 digest * all - OpenSearch/Elasticsearch name cleanup * all - cleanup nodejs dependencies * all - refactor how authentication is done, everything now uses passportjs * all - support oidc authentication method * all - caTrustFile setting should work everywhere * capture - support ERSPAN Type I and vlan for Type II * capture - new kafka plugin for sessions * capture - use malloc instead of GSlice * capture - corrupt DNS alt name memory leak fixed * capture - Added simpleFreeOutputBuffers setting * cont3xt - raw create link groups * cont3xt - two clicks to delete link groups or links * cont3xt - classify domains with multiple dashes correctly * cont3xt - added ability to copy links between link groups * cont3xt - support intl phonenumbers * db.pl - Initial OpenSearch ISM support * db.pl - Better error text for cert verify failure * esproxy - fix converting basic auth to base64 * viewer - fix field actions crash * viewer - can now use expression http.request.FIELD or http.response.FIELD with headers-http-request, headers-http-response defined fields * viewer - support viewing ipv6 DLT_RAW (#1293) * viewer - ESAdmin -> Unflood works on users cluster now also * viewer - support running in s2s auth mode only 2023-11-07T15:01:04+00:00 moloch v4.3.0 2023-11-07T15:01:54+00:00 ## Changelog * BREAKING - Only SuperAdmin can assign *Admin roles now * release - fix kafka library linking * release - al2023 support * release - improve arkime_config_interfaces.sh * release - Configure doesn't offer demo Elasticsearch on Arch * release - reqBodyOnlyUtf8=true in sample config file * all - support colon in OpenSearch/Elasticsearch password * all - fix some prototype pollution * all - improve roles enforcement * all - New authTrustProxy setting * capture - tcpClosingTimeout setting controls delay before saving tcp sessions after close * capture - default dbBulkSize to 1M, min 500K, max 15M and removed from sample config file * capture - s3 writer now writes multiple files based on packetThreads * capture - s3 writer supports zstd, s3Compression setting * capture - s3 writer compression level, s3CompressionBlockSize setting * capture - s3 writer block size, s3CompressionBlockSize setting * capture - s3 writer gap encoding, s3GapPacketPos setting * capture - s3 writer when s3UseECSEnv is true use container env vars to find the id/key/token for s3 auth * capture - improve Gh0st parser (#2225) * capture - new dnp3 & finger classifier * capture - tcphealthcheck adding debugging * capture/viewer - includes setting ignores missing files starting with - * cont3xt - add malicious tidbit from urlscan results * cont3xt - add malicious and brand columns to results table for urlscan * cont3xt - link group UI improvements * cont3xt - add createDate for whois data * db.pl - new --ifneeded option to init/upgrade that will exit if not needed * parliament - fix digest auth * parliament - better auth support * parliament - improve issue page and filters * viewer - display errors when cronQueries isn't configured * viewer - fix first sessions table row obscured sometimes * viewer - disable more apis in demo mode * viewer - allow roles forced expression without user forced expression (#2213) * viewer - s3 now use each file's bucket to determine access style * wise - only send csp headers in initial request for wise page 2023-11-07T15:01:54+00:00 moloch v4.3.1 2023-11-07T15:02:44+00:00 ## Changelog * BREAKING - If running mixed versions of Arkime, broken cron queries error might show on OLD version * release - fix ubuntu22 kafka dep * all - passwordSecret log message now has the right [section] * capture - --tags option now works as well as --tag * viewer - new auto cronQueries setting * viewer - change where primary viewer info is stored to not cause constant mapping change * viewer - fixed ipv6 not working, now assumes zero filled with mask (if not provided) * viewer - code refactor into javascript classes 2023-11-07T15:02:44+00:00 moloch v4.3.2 2023-11-07T15:03:35+00:00 ## Changelog * release - cyberchef 10.4.0 libpcap 1.10.4 * all - config 'prefix' can be at most 50 characters * all - new cookie generation code * capture - handle packets better at epoch time * cont3xt - add twilio country code tidbit * cont3xt - add httpRealm to sample config * cont3xt - help improvements * cont3xt - minor UI improvements * db.pl - set ISM deleteTime for sessions correctly * esproxy - add tests * parliament - fixed occasional missing token error * viewer/wise - Field/Value actions now support all:true to show on every instance * viewer - Fix Src/Dst mouse over for packets/bytes * viewer - Field Actions didn't work in expanded meta * viewer - Fix sending/receiving sessions not working 2023-11-07T15:03:35+00:00 moloch v4.4.0 2023-11-07T15:04:28+00:00 ## Changelog * release - cyberchef 10.5.2 * release - update arkime_update_geo.sh to use different manuf location * all - improved json verification * all - better logging when requiredAuthHeader fails * all - better role creation/usage validation * all - don't allow circular role dependencies * all - now need to be an userAdmin and *Admin to update *Admin change settings for another user * all - more auth debugging * all - can now change the password of another *Admin user if you have userAdmin and all the same *Admin * all - hide webEnable, headerAuthEnable checkboxes for roles * all - oidc now uses sameSite: Lax instead of sameSite: Strict for cookies * capture - handle tcp port reuse better * capture - fix kafka memory leak when produce fails * cont3xt - New overview cards * cont3xt - fix startup race condition with db init * cont3xt - new search protocol to prepare for bulk * parliament - fix parliament clean start not letting auth be set up * viewer - gtp decoding * viewer - demo mode improvements, arkimeAdmin can use normally * viewer - fix unique endpoint not enforcing user time limit 2023-11-07T15:04:28+00:00 moloch v4.5.0 2023-11-07T15:05:17+00:00 ## Changelog * release - node 16.20.2 * release - added missingok to default logrotate for arkime * capture - dns answers were double parsed * capture - custom-fields honors viewerOnly:true * capture - added dns.https fields * capture - added cert:certificate-authority tag (thanks mcgillowen ) * cont3xt - remove raw view button for link groups on the cont3xt search page * cont3xt - Overview shortcut * cont3xt - fixed overviews not updating on switch * db.pl - don't allow '.' to be used for sync/add path * viewer - fixed ipv6 session display issues when :: in ip * viewer - http display rewritten to not depending on nodejs internals * viewer - gpe display improvements 2023-11-07T15:05:17+00:00 moloch v4.6.0 2023-11-07T15:06:14+00:00 ### [Installation Instructions](https://raw.githubusercontent.com/arkime/arkime/main/release/README.txt) | [4.x Upgrade instructions](https://arkime.com/faq#how_do_i_upgrade_to_arkime_4) | [Copyright Notices](https://s3.amazonaws.com/files.molo.ch/NOTICE.txt) | [FAQ](https://arkime.com/faq) | [CHANGELOG](https://raw.githubusercontent.com/arkime/arkime/main/CHANGELOG) # :sparkles: What's new ✨ ## Release - curl 8.4.0 - fix viewer systemd file ## Capture - fix zstd hanging capture on full buffer ## Viewer - corrupt http session decoding might hang viewer - handle uncompressing pcap errors better - role check in UI didn't always work ## All - handle cookies encoded with bad proxy --- ### ℹ️ Download Info We offer downloads for many different OS versions because of library differences. For example, use the el7 download for Centos 7 or RHEL 7. If you have a libssl version error, it is most likely that the wrong download was used for your OS. The moloch builds have the old filesystem layouts, we will stop providing the moloch builds in 2024. 2023-11-07T15:06:14+00:00 moloch v5.0.0-rc1 2023-11-09T17:02:25+00:00 ### [Installation Instructions](https://raw.githubusercontent.com/arkime/arkime/main/release/README.txt) | [5.x Upgrade instructions](https://arkime.com/faq#how_do_i_upgrade_to_arkime_5) | [Copyright Notices](https://s3.amazonaws.com/files.molo.ch/NOTICE.txt) | [FAQ](https://arkime.com/faq) | [CHANGELOG](https://raw.githubusercontent.com/arkime/arkime/main/CHANGELOG) A db.pl upgrade is required when upgrading. # ✨ What's new ✨ ## BREAKING - #2297 s3Compression/simpleCompression now default to zstd - #2297 s3WriteGzip removed, use s3Compression=gzip - #2297 s3GapPacketPos defaults to TRUE - #2297 enablePacketDedup defaults to TRUE - #2299 authMode defaults to digest now - #2312 removed old v1 APIs - #2349 parliament password removed, must configure common auth via the UI before upgrading or manually in the config file see [parliament](https://arkime.com/settings#parliament) and [how do I upgrade to 5](https://arkime.com/faq#how_do_i_upgrade_to_arkime_5) - #2402 WISE/tagger must now use http.request.FIELD/http.response.FIELD when referencing header defined with headers-http-request/headers-http-response - #2450 Centos 7 build no longers supports pfring - #2453 Increase simpleCompressionBlockSize default to 64000 - #2299 #2308 Remove anonymous auth as the default ## Release - #2448 zstd 1.5.5, nghttp2 1.57.0, maxmind 1.7.1, yara 4.2.3 - #2448 node v18.18.2 - Centos 7, Ubuntu 18, Alpine use unofficial builds - #2447 support building on alpine ## All - #2316 programs support same config file formats (ini/json/yaml) and retrieval (file, elasticsearch) - #2419 json/yaml config file formats now allow arrays instead of comma/semi separated - #2299 #2308 authMode setting added - #2299 #2408 #2463 added authMode: basic, form, basic+form, basic+oidc, headerOnly, header+digest (same as header), header+basic - #2387 notifiers for parliament and arkime merged conflicts mitigated by appending "Parliament" to parliament notifiers - #2396 drop privileges is now AFTER http(s) list ## Capture - #2295 moloch converted to arkime - #2312 override ips can now set any field - #2312 overrideIpsFiles setting - #2314 packetDropIpsFiles setting - #2390 can have negative cert.validDays/cert.remainingDays (thanks @mcgillowen) - #2390 added cert.remainingSeconds/cert.remainingSeconds (thanks @mcgillowen) - #2390 cert.remainingDays is now based on the firstPacket of session instead of current time (thanks @mcgillowen) - #2409 JA4 support - #2409 JA3/JA4 support for smtp STARTTLS - #2297 always build zstd (except arch) ## Cont3xt - #2121 new bulk UI and support for bulk queries - #2271 lots of keyboard shortcut improvements - #2383 new array syntax for links substitution - #2382 new OpenSearch/Elasticsearch integration (config file only) - #2441 new csv/json file/url/redis integration (config file only) - #2385 new viewRoles in config file per integration to control access - #2407 transfer ownership of resources - #2437 new csv/json data source supports - #2441 new redis data source support ## ESProxy - #2483 #2484 support field updates/deletes ## Viewer - #2296 removed x-moloch-auth - #2392 files/history/stats now have cluster dropdown for multiviewer - #2402 http.request.FIELD and http.response.FIELD supported - #2404 add editor for resources - #2407 transfer ownership of resources - #2482 added uploadRoles to control who can upload ## Parliament - #2377 dashboard-only mode removed, if you want users to just see the dashboard don't assign them the parliamentUser role - #2395 configuration is now stored in opensearch/elasticsearch --- ### ℹ️ Download Info We offer downloads for many different OS versions because of library differences. For example, use the el7 download for Centos 7 or RHEL 7. If you have a libssl version error, it is most likely that the wrong download was used for your OS. The moloch builds have the old filesystem layouts, we will stop providing the moloch builds in 2024 2023-11-09T17:02:25+00:00 moloch last-commit 2023-12-11T12:49:37+00:00 Hi! After every commit to the main branch of Arkime we build and store the results here. The builds are based on Arkime 5, so if upgrading from Arkime 4, make sure you've followed the [upgrading to 5 instructions](https://arkime.com/faq#how_do_i_upgrade_to_arkime_5). We also have a [stable](https://github.com/arkime/arkime/releases/latest) release and a [5.0.0-rc2](https://github.com/arkime/arkime/releases/v5.0.0-rc2) release. [Installation Instructions](https://raw.githubusercontent.com/arkime/arkime/main/release/README.txt) | [5.x Upgrade instructions](https://arkime.com/faq#how_do_i_upgrade_to_arkime_5) | [Copyright Notices](https://s3.amazonaws.com/files.molo.ch/NOTICE.txt) | [FAQ](https://arkime.com/faq) | [CHANGELOG](https://raw.githubusercontent.com/arkime/arkime/main/CHANGELOG) The EL7 and Ubuntu 18 builds are still available [here](https://arkime.com/downloads#latest) until 3/1/2024. 2023-12-11T12:49:37+00:00 moloch v5.0.0-rc2 2024-01-09T19:43:56+00:00 ### [Installation Instructions](https://raw.githubusercontent.com/arkime/arkime/main/release/README.txt) | [5.x Upgrade instructions](https://arkime.com/faq#how_do_i_upgrade_to_arkime_5) | [Copyright Notices](https://s3.amazonaws.com/files.molo.ch/NOTICE.txt) | [FAQ](https://arkime.com/faq) | [CHANGELOG](https://raw.githubusercontent.com/arkime/arkime/main/CHANGELOG) | [JA4+ Install](https://arkime.com/ja4) A db.pl upgrade is required when upgrading from 4.x # ✨ What's new ✨ ## BREAKING - #2297 s3Compression/simpleCompression now defaults to zstd - #2297 s3WriteGzip removed, use s3Compression=gzip for gzip instead of new zstd default - #2297 s3GapPacketPos defaults to TRUE - #2297 enablePacketDedup defaults to TRUE - #2299 #2308 authMode defaults to digest now - #2312 removed old v1 viewer APIs - #2349 parliament password removed, must configure common auth via the UI before upgrading or manually in the config file see [parliament](https://arkime.com/settings#parliament) and [how do I upgrade to 5](https://arkime.com/faq#how_do_i_upgrade_to_arkime_5) - #2402 WISE/tagger must now use http.request.FIELD/http.response.FIELD when referencing header defined with headers-http-request/headers-http-response - #2450 Centos 7 build no longers includes pfring support - #2453 Increase simpleCompressionBlockSize default to 64000 ## Release - #2448 zstd 1.5.5, nghttp2 1.57.0, maxmind 1.7.1, yara 4.2.3 - #2443 Centos 7, Ubuntu 18, Alpine use unofficial builds of node - #2543 node v18.19.0 - #2447 support building on alpine - #2549 use configure prefix more places (thanks @vpiserchia) ## All - #2316 programs support same config file formats (ini/json/yaml) and retrieval (file, elasticsearch) - #2419 json/yaml config file formats now allow arrays instead of comma/semi separated - #2299 #2308 authMode setting added - #2299 #2408 #2463 added authMode: basic, form, basic+form, basic+oidc, headerOnly, header+digest (same as header), header+basic - #2387 notifiers for parliament and arkime merged conflicts mitigated by appending "Parliament" to parliament notifiers - #2396 drop privileges is now AFTER http(s) list - #2509 add optional login message for form auth - #2511 new authOIDCScope setting - #2482 new logoutUrl setting - #2571 new scheme pcap reading ## Capture - #2295 moloch converted to arkime - #2312 override ips can now set any field - #2312 overrideIpsFiles setting - #2314 packetDropIpsFiles setting - #2390 can have negative cert.validDays/cert.remainingDays (thanks @mcgillowen) - #2390 added cert.remainingSeconds/cert.remainingSeconds (thanks @mcgillowen) - #2390 cert.remainingDays is now based on the firstPacket of session instead of current time (thanks @mcgillowen) - #2409 JA4 support - #2409 JA3/JA4 support for smtp STARTTLS - #2297 always build zstd (except arch) - #2517 new custom-fields-remap feature - #2186 count the number of http methods per session - #2528 new oui.txt location, some names have changes, fixes #2347 - #2539 new tls:has_esni tag if the client hello has esni - #2553 fix rules range matching not working always - #2554 support fieldSet tcpflag rules - #2576 support different dlt for pcap-over-ip ## Cont3xt - #2121 new bulk UI and support for bulk queries - #2271 lots of keyboard shortcut improvements - #2383 new array syntax for links substitution - #2382 new OpenSearch/Elasticsearch integration (config file only) - #2441 new csv/json file/url/redis integration (config file only) - #2385 new viewRoles in config file per integration to control access - #2407 transfer ownership of resources - #2437 new csv/json data source supports - #2441 new redis data source support - #2507 demoMode added - #2527 skipChildren added - #2532 new wise integration ## ESProxy - #2483 #2484 support field updates/deletes ## Viewer - #2296 removed x-moloch-auth - #2392 files/history/stats now have cluster dropdown for multiviewer - #2402 http.request.FIELD and http.response.FIELD supported - #2404 add editor for resources - #2407 transfer ownership of resources - #2482 added uploadRoles to control who can upload - #2501 add defaultTimeRange setting - #2521 add footerTemplate setting - #2525 add [config setting](https://arkime.com/settings#spiViewCategoryOrder) to set spiview category order - #2523 resize session detail field label/values - #2552 added %URIEncodedText% for URI encoded substitution (thanks @vpiserchia) ## Parliament - #2377 dashboard-only mode removed, if you want users to just see the dashboard don't assign them the parliamentUser role - #2395 configuration is now stored in opensearch/elasticsearch - #2530 add Users page ## WISE - #2537 new urlScrapePrefix/urlScrapeSuffix used with urlScrapeRedirect - #2537 new jsonl format supported ### Download Info We offer downloads for many different OS versions because of library differences. For example, use the el7 download for Centos 7 or RHEL 7. If you have a libssl version error, it is most likely that the wrong download was used for your OS. The moloch builds have the old filesystem layouts, we will stop providing the moloch builds in 2024 2024-01-09T19:43:56+00:00