# ─────────────────────────────────────────────────────────────────────────────────────────────────── # ─██████████████─████████████████───██████──────────██████─██████████████─██████████─██████████████─ # ─██░░░░░░░░░░██─██░░░░░░░░░░░░██───██░░██████████████░░██─██░░░░░░░░░░██─██░░░░░░██─██░░░░░░░░░░██─ # ─██░░██████░░██─██░░████████░░██───██░░░░░░░░░░░░░░░░░░██─██░░██████░░██─████░░████─██░░██████████─ # ─██░░██──██░░██─██░░██────██░░██───██░░██████░░██████░░██─██░░██──██░░██───██░░██───██░░██───────── # ─██░░██████░░██─██░░████████░░██───██░░██──██░░██──██░░██─██░░██████░░██───██░░██───██░░██████████─ # ─██░░░░░░░░░░██─██░░░░░░░░░░░░██───██░░██──██░░██──██░░██─██░░░░░░░░░░██───██░░██───██░░░░░░░░░░██─ # ─██░░██████░░██─██░░██████░░████───██░░██──██████──██░░██─██░░██████░░██───██░░██───██░░██████████─ # ─██░░██──██░░██─██░░██──██░░██─────██░░██──────────██░░██─██░░██──██░░██───██░░██───██░░██───────── # ─██░░██──██░░██─██░░██──██░░██████─██░░██──────────██░░██─██░░██──██░░██─████░░████─██░░██───────── # ─██░░██──██░░██─██░░██──██░░░░░░██─██░░██──────────██░░██─██░░██──██░░██─██░░░░░░██─██░░██───────── # ─██████──██████─██████──██████████─██████──────────██████─██████──██████─██████████─██████───────── # ─────────────────────────────────────────────────────────────────────────────────────────────────── [sources.if_{{pk}}_socket_source] type = "syslog" address = "0.0.0.0:{{port}}" mode = "udp" # 1. Check is message CEF [transforms.if_{{pk}}_cef_filter] type="filter" inputs=["if_{{pk}}_socket_source"] condition = ''' match(string!(.message),r'^.*CEF:.*')''' # Parse data from socket [transforms.if_{{pk}}_parse_logs] type = "remap" inputs = ["if_{{pk}}_cef_filter"] source = ''' . |= parse_regex!(.message,r'CEF:\d+\|(?P([^\|]*))\|(?P([^\|]*))\|(?P([^\|]*))\|(?P([^\|]*))\|(?P([^\|]*))\|(?P([^\|]*))\|(?P.*)') .aggregated = "false" .orig_message = .message .event_src_msg = .orig_message .format = "CEF" .type="armaif_1" .extension =strip_whitespace(.extension) .extension = strip_ansi_escape_codes(.extension) .source_ip= .host .destination_ip = "127.0.0.1" .@timestamp = now() ''' #Check device product [transforms.if_{{pk}}_check_device_product] type="lua" version="2" inputs=["if_{{pk}}_parse_logs"] hooks.process =""" function(event,emit) if event.log["device_product"] == "ARMAIF" then event.log.type="armaif_{{ pk }}" event.log.device_product = "Industrial Firerwall" event.log.message = "Empty message" event.log.source_host = "0.0.0.0" event.log.destination_host = "localhost" emit(event) end end """ #Parse key value [transforms.if_{{pk}}_parse_key_value] type="lua" version="2" inputs=["if_{{pk}}_check_device_product"] hooks.process=""" function(event,emit) str=event.log["extension"] for name, value in str:gmatch"%s*([^=]*)=([^=]*)%f[%s%z]" do event.log[name]=value end emit(event) end """ {% if adjust_datetime == 1 %} [transforms.if_{{pk}}_made_timestamp] type="remap" inputs=["if_{{pk}}_parse_key_value"] source =""" .event_timestamp= now() .event_timestamp = format_timestamp!(.event_timestamp, format: "%+") """ {%else%} [transforms.if_{{pk}}_made_timestamp] type="lua" version="2" inputs=["if_{{pk}}_parse_key_value"] hooks.process = """ function(event,emit) local date_time = tonumber(event.log["rt"]) m_date_time = date_time/1000 event.log.event_timestamp = os.date("!%Y-%m-%dT%H:%M:%SZ",m_date_time) emit(event) end """ {%endif%} #Parse input ARPWATCH [transforms.if_{{pk}}_parse_arpwatch] type="lua" inputs =["if_{{pk}}_made_timestamp"] version= "2" hooks.process =""" function(event,emit) if event.log["name"] == "Arpwatch alert" then if event.log["act"] ~= nil then event.log.device_action = event.log["act"] event.log.sign_id = event.log["act"] end event.log.event_severity = event.log["severity"] event.log.sign_subcategory = event.log["signature"] event.log.sign_category = "ARPWATCH" event.log.source_ip = event.log["src"] event.log.sign_name = "New device "..event.log["src"] end emit(event) end """ # Parse input firewall [transforms.if_{{pk}}_parse_firewall] type="lua" version = "2" inputs = ["if_{{pk}}_parse_arpwatch"] hooks.process = """ function (event,emit) if event.log["name"] == "PF rule alert" then if event.log["act"] ~= nil then event.log.device_action = event.log["act"] end event.log.id = "armaif_1_firewall_match" event.log.event_severity = event.log["severity"] event.log.event_src_msg = event.log["message"] event.log.event_protocol = event.log["proto"] event.log.sign_id = event.log["cs1"] event.log.sign_name = "Firewall Rule" event.log.sign_category = "PF" event.log.source_ip = event.log["src"] event.log.source_port = event.log["spt"] event.log.destination_ip = event.log["dst"] event.log.destination_port = event.log["dpt"] event.log.celery_done = "false" end emit(event) end """ # Parse input NTP [transforms.if_{{pk}}_parse_ntp] type="lua" version="2" inputs=["if_{{pk}}_parse_firewall"] hooks.process = """ function (event,emit) if event.log["signature"] == "ntppower" and event.log["name"] == "Ntp power" then event.log.id = "armaif_{{ pk }}_ntp_match" event.log.event_severity = event.log["severity"] event.log.event_src_msg = event.log["message"] event.log.device_product = event.log["deviceFacility"] event.log.device_action = event.log["act"] event.log.sign_id = event.log["dvcpid"] event.log.sign_name = event.log["signature"] event.log.sign_category = "NTP" event.log.celery_done = "false" event.log.source_ip= event.log["host"] end emit(event) end """ #Parse input Suricata 1 [transforms.if_{{pk}}_parse_suricata_1] type="lua" version="2" inputs=["if_{{pk}}_parse_ntp"] hooks.process = """ function (event,emit) if event.log["signature"] == "idspower" then if event.log["classification"] ~= nill then event.log.sign_subcategory = event.log["classification"] end event.log.id = "armaif_1_suricata_match" event.log.event_severity = event.log["severity"] if event.log["msg"] ~= nil then event.log.message = event.log["msg"] end event.log.device_vendor = event.log["device_vendor"] event.log.device_product = event.log["device_product"] event.log.device_version = event.log["device_version"] event.log.event_protocol = event.log["proto"] event.log.device_action = event.log["act"] event.log.sign_id = event.log["cs1"] event.log.sign_name = event.log["name"] event.log.sign_category = "IDS" if event.log["ip_src"] ~= nil then event.log.source_ip = event.log["ip_src"] end event.log.source_port = event.log["port_src"] if event.log["ip_dst"] ~=nil then event.log.destination_ip = event.log["ip_dst"] end event.log.destination_port = event.log["port_dst"] event.log.celery_done = "false" end emit(event) end """ # Parse input Suricatqa 2 [transforms.if_{{pk}}_parse_suricata_2] type="lua" version="2" inputs=["if_{{pk}}_parse_suricata_1"] hooks.process = """ function (event,emit) if (event.log["device_product"] == "Suricata" and event.log["signature"] ~= "Turn off") then event.log.id = "armaif_1_suricata_match_2" event.log.event_severity = event.log["severity"] if event.log["msg"] ~= nil then event.log.message = event.log["msg"] end event.log.event_protocol = event.log["proto"] event.log.device_vendor = event.log["device_vendor"] event.log.device_product = event.log["device_product"] event.log.device_version = event.log["device_version"] event.log.sign_id = event.log["signature"] event.log.sign_name = event.log["name"] event.log.sign_category = "IDS" event.log.source_ip = event.log["ip_src"] event.log.source_port = event.log["port_src"] event.log.destination_ip = event.log["ip_dst"] event.log.destination_port = event.log["port_dst"] event.log.celery_done = "false" end emit(event) end """ #Parse input Suricata 3 [transforms.if_{{pk}}_parse_suricata_3] type="lua" version="2" inputs=["if_{{pk}}_parse_suricata_2"] hooks.process = """ function(event,emit) if event.log["signature"] == "idsalert" then if event.log["classification"] ~= nil then event.log.sign_subcategory = event.log["classification"] end event.log.id = "armaif_{{ pk }}_suricata_match_3" event.log.event_severity = event.log["severity"] if event.log["msg"] ~= nil then event.log.message = event.log["msg"] end event.log.event_protocol = event.log["proto"] event.log.device_vendor = event.log["device_vendor"] event.log.device_product = event.log["device_product"] event.log.device_version = event.log["device_version"] event.log.sign_id = event.log["act"] event.log.sign_name = event.log["signature"] event.log.sign_category = "IDS" if event.log["src"] ~=nil then event.log.source_ip = event.log["src"] end event.log.source_port = event.log["spt"] if event.log["dst"] ~= nil then event.log.destination_ip = event.log["dst"] end event.log.destination_port = event.log["dpt"] event.log.celery_done = "false" end emit(event) end """ #Parse input Suricata 4 [transforms.if_{{pk}}_parse_suricata_4] type="lua" version="2" inputs=["if_{{pk}}_parse_suricata_3"] hooks.process= """ function(event,emit) if (event.log["device_product"] == "Suricata" and event.log["signature"] == "Turn off") then event.log.id = "armaif_{{ pk }}_suricata_match_4" event.log.event_severity = event.log["severity"] if event.log["msg"] ~= nil then event.log.message = event.log["msg"] end event.log.event_protocol = "NULL" event.log.device_vendor = event.log["device_vendor"] event.log.device_product = event.log["device_product"] event.log.device_version = event.log["device_version"] event.log.device_action = event.log["act"] event.log.sign_id = event.log["signature"] event.log.sign_name = event.log["name"] event.log.sign_category = "IDS" event.log.source_ip = "127.0.0.1" event.log.source_port = "0" event.log.destination_ip = "127.0.0.1" event.log.destination_port = "0" event.log.celery_done = "false" end emit(event) end """ #Parse input Suricata 5 [transforms.if_{{pk}}_parse_suricata_5] type="lua" version="2" inputs=["if_{{pk}}_parse_suricata_4"] hooks.process= """ function(event,emit) if event.log["signature"] == "integrityalert" then event.log.id = "armaif_{{pk}}_suricata_match_5" event.log.event_severity = event.log["severity"] if event.log["msg"] ~= nil then event.log.message = event.log["msg"] end event.log.device_vendor = event.log["device_vendor"] event.log.device_product = event.log["device_product"] event.log.device_version = event.log["device_version"] event.log.sign_id = "integrityalert" event.log.sign_name = event.log["signature"] event.log.sign_category = "Integrity" event.log.celery_done = "false" end emit(event) end """ # Parse input Web access [transforms.if_{{pk}}_parse_web_access] type ="lua" version="2" inputs=["if_{{pk}}_parse_suricata_5"] hooks.process= """ function(event,emit) if event.log["signature"] == "accessalert" then event.log.id = "armaif_{{ pk }}_awb_access_match" event.log.event_severity = event.log["severity"] event.log.event_src_msg = event.log["message"] if event.log["msg"] ~= nil then event.log.message = event.log["msg"] end event.log.event_protocol = event.log["app"] event.log.device_vendor = event.log["device_vendor"] event.log.device_product = event.log["device_product"] event.log.device_version = event.log["device_version"] event.log.device_action = event.log["act"] event.log.sign_id = event.log["signature"] event.log.sign_category = "HTTP" event.log.sign_subcategory = "Access" event.log.sign_name = event.log["name"] event.log.source_ip = event.log["src"] event.log.destination_ip = event.log["dst"] event.log.celery_done = "false" end emit(event) end """ #Parse input Web Auth [transforms.if_{{pk}}_parse_web_auth] type="lua" version="2" inputs=["if_{{pk}}_parse_web_access"] hooks.process = """ function(event,emit) if ( event.log["signature"] == "webauth" and event.log["name"] == "Web authentication") then event.log.id = "armaif_{{ pk }}_web_auth_match" event.log.event_severity = event.log["severity"] event.log.event_src_msg = event.log["message"] event.log.device_vendor = event.log["device_vendor"] event.log.device_product = event.log["name"] event.log.device_version = event.log["device_version"] event.log.device_action = "Auth" event.log.sign_id = event.log["signature"] event.log.sign_name = event.log["name"] event.log.sign_category = "HTTP" event.log.sign_subcategory = "Auth" event.log.source_ip = event.log["src"] event.log.source_user = event.log["suser"] event.log.celery_done = "false" end emit(event) end """ #Parse input lighttpdaccess [transforms.if_{{pk}}_parse_lighttpdaccess] type="lua" version="2" inputs=["if_{{pk}}_parse_web_auth"] hooks.process= """ function(event,emit) if event.log["signature"] == "lighttpdaccess" then event.log.id = "armaif_{{ pk }}_lhttp_match" event.log.event_severity = event.log["severity"] event.log.event_src_msg = event.log["message"] event.log.device_vendor = event.log["device_vendor"] event.log.device_product = event.log["device_product"] event.log.device_version = event.log["device_version"] event.log.sign_id = event.log["signature"] event.log.sign_name = event.log["name"] event.log.sign_category = "HTTP" event.log.sign_subcategory = "Auth" event.log.source_ip = event.log["src"] event.log.destination_ip = event.log["dst"] event.log.celery_done = "false" end emit(event) end """ #Parce ClamAv [transforms.if_{{pk}}_parse_clam] type="lua" version="2" inputs=["if_{{pk}}_parse_lighttpdaccess"] hooks.process= """ function(event,emit) if event.log["signature"] == "clamav_alert" then event.log.id = "armaif_{{ pk }}_clam_match" event.log.event_severity = event.log["severity"] event.log.event_src_msg = event.log["message"] event.log.request_url=event.log["cs1"] event.log.sign_name='CLAMAV alert' event.log.sign_category='HTTP' event.log.virus_name = event.log["cs2"] event.log.device_action = event.log["act"] event.log.source_ip = event.log["src"] event.log.celery_done = "false" end emit(event) end """ #Parse NTP sync [transforms.if_{{pk}}_parse_ntpsync] type="lua" version="2" inputs=["if_{{pk}}_parse_clam"] hooks.process= """ function(event,emit) if event.log["signature"] == "ntpmanualsync" then event.log.id = "armaif_{{ pk }}_ntpsync_match" event.log.event_severity = event.log["severity"] event.log.event_src_msg = event.log["message"] event.log.device_product = event.log["deviceFacility"] event.log.message = event.log["msg"] event.log.sign_name=event.log["name"] event.log.sign_category='NTP' event.log.attempts_count = event.log["cs1"] event.log.celery_done = "false" end emit(event) end """ #Create UUID [transforms.if_{{pk}}_create_uuid] type = "remap" inputs=["if_{{pk}}_parse_ntpsync"] source = """ .event_id = uuid_v4() """ # Replace [transforms.if_{{pk}}_replace] type="lua" version="2" inputs=["if_{{pk}}_create_uuid"] hooks.process = """ function(event,emit) event.log.event_src_msg = event.log["orig_message"]:gsub("\u0000","") event.log.orig_message = event.log["message"] emit(event) end """ # Delete bad fields [transforms.if_{{pk}}_prune_fields] type="lua" version="2" inputs=["if_{{pk}}_replace"] source =""" function check_field(field) local fields_list = {'destination_ip', 'source_user', 'event_severity', 'event_protocol', 'device_vendor', 'event_src_msg', 'sign_name', 'sign_subcategory', 'source_ip', 'event_id', 'device_version', 'destination_port', 'device_product', 'device_action', 'sign_id', 'message', 'source_port', 'sign_category', 'event_timestamp', '@timestamp', 'format', 'type', 'source_host', 'destination_host', 'request_url', 'virus_name', } for key,value in pairs(fields_list) do if value == field then return true end end return false end function process (event,emit) for f, v in pairs(event.log) do if check_field(f) ~= true then event.log[f] = nil end end emit(event) end """ hooks.process="process" # Cast variables to the right types [transforms.if_{{pk}}_cast_types] type="remap" inputs = ["if_{{pk}}_prune_fields"] source = ''' .event_uuid = .event_id .source_port = to_int!(.source_port) .destination_port = to_int!(.destination_port) .aggregated = to_bool!(.aggregated) .event_severity = to_int!(.event_severity) .event_timestamp = to_timestamp!(.event_timestamp) .@timestamp = to_timestamp!(.@timestamp) ''' [transforms.if_{{pk}}_delete_null_values] type="lua" version="2" inputs = ["if_{{pk}}_cast_types"] hooks.process = """ function(event,emit) if event.log["source_port"] == 0 or event.log["source_port"] == "0" then event.log.source_port = nil end if event.log["destination_port"] == 0 or event.log["destination_port"] == "0" then event.log.destination_port = nil end emit(event) end """ ### Print parsed logs to stdout #[sinks.print_{{pk}}] # type = "console" # inputs=["if_{{pk}}_delete_null_values"] # encoding.codec ="json" # [sinks.if_{{pk}}_elasticsearch_vector] type = "elasticsearch" inputs = ["if_{{pk}}_delete_null_values"] compression = "none" healthcheck = true auth.strategy= "basic" auth.user = "{{ elastic_login }}" auth.password = "{{ elastic_pass }}" endpoint = "{{ elastic_url }}" normal.index = "arma-%Y.%m.%d" id_key = "event_uuid"