DNS is a high volume low latency datagram protocol at the heart of the Internet -- it enables almost all other traffic flows. Any analysis of network traffic for security purposes will necessarily include contemporaneous DNS traffic which might have resulted from or directed that traffic. Netflow by itself can answer the question, "what happened?" but it cannot by itself answer the equally important question, "why?"
Collecting DNS query and response data has always been challenging due to the impedance mismatch between DNS as an asynchronous datagram service and available synchronous persistent storage systems. Success in DNS telemetry has historically come from the PCAP/BPF approach, where the collection agent reassembles packets seen 'on the wire' into DNS transaction records, with complete asynchrony from the DNS server itself. It is literally and always preferable to drop transactions from the telemetry path than to impact the operation a production DNS server in any way.
BPF/PCAP is not a panacea, though, since the complexity of state-keeping means that most passive DNS collectors are blind to TCP transactions, and all are blind to data elements which don't appear on the wire, such as cache purge or cache expiration events, or to "view" identifiers or current delegation point. The Farsight Security team has therefore designed a new open source and open protocol system called 'dnstap' with a transmission/reception paradigm that preserves the necessary lossiness of DNS transaction collection while avoiding the state-keeping of BPF/PCAP based systems.
This talk will cover passive DNS including collection, sharing, post-processing, database construction, and access, using the Farsight Security system as a model. 'dnstap' will be introduced in that context, including a status report and road-map.