Intellectual Property and the Internet/Deep packet inspection

Deep Packet Inspection (DPI) (also called complete packet inspection and Information eXtraction - IX -) is a form of computer network packet filtering that examines the data part (and possibly also the header) of a packet as it passes an inspection point, searching for protocol non-compliance, viruses, spam, intrusions or predefined criteria to decide if the packet can pass or if it needs to be routed to a different destination, or for the purpose of collecting statistical information. There are multiple headers for IP packets, network equipment only needs to use the first of these (the IP header) for normal operation, but use of the second header (TCP, UDP etc.) is normally considered to be shallow packet inspection (usually called Stateful Packet Inspection) despite this definition.http://www.securityfocus.com/infocus/1817 The Perils of Deep Packet Inspection Security Focus Dr. Thomas Porter 2005-01-11 2008-03-02 Deep Packet Inspection (and filtering) enables advanced network management, user service, and security functions as well as internet data mining, eavesdropping, and censorship. Although DPI technology has been used for Internet management for many years, some advocates of net neutrality fear that the technology can be used anticompetitively or to reduce the openness of the Internet.http://dpi.priv.gc.ca/index.php/essays/just-deliver-the-packets/ Just Deliver the Packets, in: "Essays on Deep Packet Inspection", Ottawa Office of the Privacy Commissioner of Canada Hal Abelson, Ken Ledeen, Chris Lewis 2009 2010-01-08 DPI is currently being used by the enterprise, service providers and governments in a wide range of applications.http://userpage.fu-berlin.de/~bendrath/Paper_Ralf-Bendrath_DPI_v1-5.pdf Global technology trends and national regulation: Explaining Variation in the Governance of Deep Packet Inspection, Paper presented at the International Studies Annual Convention, New York City, 15–18 February 2009 International Studies Association Ralf Bendrath 2009-03-16 2010-01-08 Background DPI combines the functionality of an Intrusion Detection System (IDS) and an Intrusion Prevention System (IPS) with a traditional stateful firewall.http://www.securityfocus.com/infocus/1716 Firewall Evolution - Deep Packet Inspection Security Focus Ido Dubrawsky 2003-07-29 2008-03-02 This combination makes it possible to detect certain attacks that neither the IDS/IPS nor the stateful firewall can catch on their own. Stateful firewalls, while able to see the beginning and end of a packet flow, cannot on their own catch events that would be out of bounds for a particular application. While IDSs are able to detect intrusions, they have very little capability in blocking such an attack. DPIs are used to prevent attacks from viruses and worms at wire speeds. More specifically, DPI can be effective against buffer overflow attacks, Denial of Service (DoS) attacks, sophisticated intrusions, and a small percentage of worms that fit within a single packet. DPI-enabled devices have the ability to look at Layer 2 and beyond Layer 3 of the OSI model, in cases DPI can be evoked to look through Layer 2-7 of the OSI model. This includes headers and data protocol structures as well as the actual payload of the message. DPI functionality is evoked when a device looks or takes other action based on information beyond Layer 3 of the OSI model. DPI can identify and classify traffic based on a signature database that includes information extracted from the data part of a packet, allowing finer control than classification based only on header information. End points can utilize encryption and obfuscation techniques to evade DPI actions in many cases. A classified packet can be redirected, marked/tagged (see quality of service), blocked, rate limited, and of course reported to a reporting agent in the network. In this way, HTTP errors of different classifications may be identified and forwarded for analysis. Many DPI devices can identify packet flows (rather than packet-by-packet analysis), allowing control actions based on accumulated flow information. DPI at the enterprise Initially security at the enterprise was just a perimeter discipline, with a dominant philosophy of keeping unauthorized users out, and shielding authorized users from the outside world. The most frequently used tool for accomplishing this has been a stateful firewall. It can permit fine-grained control of access from the outside world to pre-defined destinations on the internal network, as well as permitting access back to other hosts only if a request to the outside world has been made previously.http://www.itbusinessedge.com/item/?ci=35275 The Case for Deep Packet Inspection IT Business Edge Elan Amir 2007-10-29 2008-03-02 However, vulnerabilities exist at network layers that are not visible to a stateful firewall. Also, an increase in the use of laptops in the enterprise makes it more difficult to prevent threats such as viruses, worms and spyware from penetrating the corporate network, as many users will connect the laptop to less-secure networks such as home broadband connections or wireless networks in public locations. Firewalls also do not distinguish between permitted and forbidden uses of legitimately-accessed applications. DPI enables IT administrators and security officials to set policies and enforce them at all layers, including the application and user layer to help combat those threats. Deep Packet Inspection is able to detect a few kinds of buffer overflow attacks. DPI can be used by the enterprise for Data Leak Prevention (DLP). When an e-mail user tries to send a protected file he may be given information on how to get the proper clearance to send the file.August 2010December 2011http://searchnetworking.techtarget.com/news/article/0,289142,sid7_gci1335767,00.html Data leak prevention starts with trusting your users SearchNetworking.com Michael Morisy 2008-10-23 2010-02-01 DPI at network/Internet service providers In addition to using DPI to secure their internal networks, Internet service providers also apply this technology on the public networks provided to customers. Common uses of DPI by ISPs are lawful intercept, policy definition and enforcement, targeted advertising, quality of service, offering tiered services, and copyright enforcement. Lawful interception Service providers are required by almost all governments worldwide to enable lawful intercept capabilities. Decades ago in a legacy telephone environment, this was met by creating a traffic access point (TAP) using an intercepting proxy server that connects to the government's surveillance equipment. This is not possible in contemporary digital networks. The acquisition component of this functionality can be provided in many ways, including DPI, DPI enabled products that are "LI or CALEA-compliant" can be used - when directed by a court order - to access a user's datastream.http://arstechnica.com/articles/culture/Deep-packet-inspection-meets-net-neutrality.ars Deep Packet Inspection meets 'Net neutrality, CALEA ars technica Nate Anderson 2007-07-25 2006-02-06 Policy definition and enforcement Service providers obligated by the service level agreement with their customers to provide a certain level of service, and at the same time enforce an acceptable use policy, may make use of DPI to implement certain policies that cover copyright infringements, illegal materials, and unfair use of bandwidth. In some countries the ISPs are required to perform filtering depending on the country's laws. DPI allows service providers to "readily know the packets of information you are receiving online—from e-mail, to websites, to sharing of music, video and software downloads".http://www.thenation.com/doc/20060213/chester The End of the Internet? The Nation Jeff Chester 2006-02-01 2006-02-06 Policies can be defined that allow or disallow connection to or from an IP address, certain protocols, or even heuristics that identify a certain application or behavior. Targeted advertising Because ISPs route all of their customers' traffic, they are able to monitor web-browsing habits in a very detailed way allowing them to gain information about their customers' interests, which can be used by companies specializing in targeted advertising. At least 100,000 US customers are tracked this way, and as many of 10% of US customers have been tracked in this way.May 2010 Technology providers include NebuAd, Front Porch and Phorm. US ISPs monitoring their customers include Knology,http://connect.charter.com/landing/op1.html Charter Communications: Enhanced Online Experience 2008-05-14 and Wide Open West. In addition, the UK ISP British Telecom has admitted testing technology from Phorm without their customers' knowledge or consent.http://www.washingtonpost.com/wp-dyn/content/article/2008/04/03/AR2008040304052.html Every Click You Make: Internet Providers Quietly Test Expanded Tracking of Web Use to Target Advertising 2008-04-04 Peter Whoriskey 2008-04-08 The Washington Post Quality of service Applications such as peer-to-peer (P2P) traffic present increasing problems for broadband service providers. P2P traffic is typically used by applications that do file sharing. This can be documents, music and videos. Due to the frequently large size of media files being transferred, P2P drives increasing traffic loads, requiring additional network capacity. Service providers say a minority of users generate large quantities of P2P traffic and degrade performance for the majority of broadband subscribers using applications such as email or Web browsing which use less bandwidth.http://www.lightreading.com/insider/details.asp?sku_id=1221&skuitem_itemid=957 Deep Packet Inspection: Taming the P2P Traffic Beast Light Reading 2008-03-03 Poor network performance increases customer dissatisfaction and leads to a decline in service revenues. DPI allows the operators to oversell their available bandwidth while ensuring equitable bandwidth distribution to all users by preventing network congestion. Additionally, a higher priority can be allocated to a VoIP or video conferencing call which requires low latency versus web browsing which does not.http://www.computerworld.com/action/article.do?command=viewArticleBasic&taxonomyId=16&articleId=9036959&intsrc=hm_topic Ball State uses Deep Packet Inspection to ensure videoconferencing performance Computer World Matt Hamblen 2007-09-17 2008-03-03 This is the approach that service providers use to dynamically allocate bandwidth according to traffic that is passing through their networks. Other Vendors claim that DPI is ineffective against P2P and that other methods of Bandwidth Management are more effective.December 2011December 2011July 2008 Tiered services Mobile and broadband service providers use DPI as a means to implement tiered service plans, to differentiate "walled garden" services from "value added", “all-you-can-eat" and "one-size-fits-all” data services.http://news.moneycentral.msn.com/ticker/article.aspx?Feed=PR&Date=20080205&ID=8139811&Symbol=ALLT Allot Deploys DPI Solution at Two Tier 1 Mobile Operators to Deliver Value- Added and Tiered Service Packages Money Central 2008-02-05 2008-03-03 By being able to charge for a "walled garden", per application, per service, or "all-you-can-eat" rather than a "one-size-fits-all" package, the operator can tailor his offering to the individual subscriber and increase their Average Revenue Per User (ARPU). A policy is created per user or user group, and the DPI system in turn enforces that policy, allowing the user access to different services and applications. Copyright enforcement ISPs are sometimes requested by copyright owners or required by courts or official policy to help enforce copyrights. In 2006, one of Denmark's largest ISPs, Tele2, was given a court injunction and told it must block its customers from accessing The Pirate Bay, a launching point for BitTorrent.http://www.infoworld.com/article/08/02/13/Danish-ISP-prepares-to-fight-Pirate-Bay-injunction_1.html Danish ISP prepares to fight Pirate Bay injunction IDG News Service Jeremy Kirk 2008-02-13 2008-03-12 Instead of prosecuting file sharers one at a time,http://www.enn.ie/frontpage/news-9617239.html Eircom and BT won't oppose music firms ENN Matthew Clark 2005-07-05 2008-03-12 October 2010 H3llBot the International Federation of the Phonographic Industry (IFPI) and the big four record labels EMI, Sony BMG, Universal Music and Warner Music have begun suing ISPs like Eircom for not doing enough about protecting their copyrights.http://arstechnica.com/news.ars/post/20080311-year-of-filters-turning-into-year-of-lawsuits-against-isps.html "Year of filters" turning into year of lawsuits against ISPs ars technica Eric Bangeman 2008-03-11 2008-03-12 The IFPI wants ISPs to filter traffic to remove illicitly uploaded and downloaded copyrighted material from their network, despite European directive 2000/31/EC clearly stating that ISPs may not be put under a general obligation to monitor the information they transmit and directive 2002/58/EC granting European citizens a right to privacy of communications. The Motion Picture Association of America (MPAA) which enforces movie copyrights, on the other hand has taken the position with the Federal Communications Commission (FCC) that network neutrality could hurt anti-piracy technology such as Deep Packet Inspection and other forms of filtering.http://www.news.com/8301-10784_3-9746938-7.html MPAA: Net neutrality could hurt antipiracy tech CNET News Anne Broach 2007-07-19 2008-03-12 Statistics DPI allows ISPs to gather statistical information about usage patterns by user group. For instance, it might be of interest whether users with a 2 Mbit connection use the network in a dissimilar manner to users with a 5 Mbit connection. Access to trend data also help network planning. December 2011<!- also wouldn't shallow packet inspection be sufficient?> Deep Packet Inspection by governments network surveillance censorship In addition to using DPI for the security of their own networks, governments in North America, Europe and Asia use DPI for various purposes such as surveillance and censorship; many of these programs are classified.http://www.networkworld.com/newsletters/isp/2007/0625isp1.html OEM provider Bivio targets government market Network World Carolyn Duffy Marsan 2007-06-27 2008-03-13 United States FCC adopts Internet CALEA requirements. The FCC, pursuant to its mandate from the US Congress, and in line with the policies of most countries worldwide, has required that all telecommunication providers, including Internet services, be capable of supporting the execution of a court order to provide real-time communication forensics of specified users. In 2006, the FCC adopted new Title 47, Subpart Z, rules requiring Internet Access Providers meet these requirements. DPI was one of the platforms essential to meeting this requirement and has been deployed for this purpose throughout the U.S. NSA warrantless surveillance controversy The National Security Agency (NSA), with cooperation from AT&T, has used Deep Packet Inspection technology to make internet traffic surveillance, sorting and forwarding more intelligent. The DPI is used to find which packets are carrying e-mail or a Voice over Internet Protocol (VoIP) phone call.http://www.nerdylorrin.net/jerry/politics/Warrantless/WarrantlessFACTS.html How the NSA warrantless wiretap system works J. I. Nelson, Ph.D. 2006-09-26 2008-03-03 Traffic associated with AT&T’s Common Backbone was "split" between two fibers, dividing the signal so that 50 percent of the signal strength went to each output fiber. One of the output fibers was diverted to a secure room; the other carried communications on to AT&T’s switching equipment. The secure room contained Narus traffic analyzers and logic servers; Narus states that such devices are capable of real-time data collection (recording data for consideration) and capture at 10 gigabits per second. Certain traffic was selected and sent over a dedicated line to a "central location" for analysis. According to Marcus’s affidavit, the diverted traffic "represented all, or substantially all, of AT&T’s peering traffic in the San Francisco Bay area," and thus, "the designers of the ... configuration made no attempt, in terms of location or position of the fiber split, to exclude data sources y comprised primarily of domestic data." Bellovin Steven M. Steven M. Bellovin Matt Blaze, Whitfield Diffie, Susan Landau, Peter G. Neumann, and Jennifer Rexford Risking Communications Security: Potential Hazards of the Protect America Act IEEE Security and Privacy 6 1 24–33 IEEE Computer Society January/February 2008 http://www.crypto.com/papers/paa-ieee.pdf 10.1109/MSP.2008.17 2008-03-03 Narus's Semantic Traffic Analyzer software which runs on IBM or Dell Linux servers, using DPI technology, sorts through IP traffic at 10Gbit/s to pick out specific messages based on a targeted e-mail address, IP address or, in the case of VoIP, phone number.http://www.wired.com/science/discoveries/news/2006/05/70914 The Ultimate Net Monitoring Tool Wired Robert Poe 2006-05-17 2008-03-03 President George W. Bush and Attorney General Alberto R. Gonzales have asserted that they believe the president has the authority to order secret intercepts of telephone and e-mail exchanges between people inside the United States and their contacts abroad without obtaining a FISA warrant.http://www.washingtonpost.com/wp-dyn/content/article/2006/01/06/AR2006010601772.html Report Rebuts Bush on Spying - Domestic Action's Legality Challenged Carol D. Leonnig The Washington Post 2007-01-07 2008-03-03 The Defense Information Systems Agency has developed a sensor platform that uses Deep Packet Inspection.https://www.dpacket.org/articles/deep-security-disa-beefs-security-deep-packet-inpection-ip-transmissions Deep Security: DISA Beefs Up Security with Deep Packet Inpection of IP Transmissions Cheryl Gerber 2008-09-18 2008-10-30 China Internet censorship in the People's Republic of China The Chinese government uses Deep Packet Inspection to monitor and censor network traffic and content that it claims harmful to Chinese citizens or state interests. This material includes pornography, information on religion, and political dissent.http://www.businessweek.com/technology/content/jan2006/tc20060112_434051.htm The Great Firewall of China Business Week Ben Elgin and Bruce Einhorn 2006-01-12 2008-03-13 Chinese network ISPs use DPI to see if there's any sensitive keyword going through their network. If so, the connection will be cut. People within China often find themselves blocked while accessing Web sites containing content related to Taiwanese and Tibetan independence, Falun Gong, the Dalai Lama, the Tiananmen Square protests and massacre of 1989, political parties that oppose that of the ruling Communist party, or a variety of anti-Communist movementshttp://www.opennetinitiative.net/studies/china/ Internet Filtering in China in 2004-2005: A Country Study Open Net Initiative 2008-03-13 as those materials were signed as DPI sensitive keywords already. China also blocks VoIP traffic in and out of their countryIt doesn't seemed to be true in China, so please give citation or delete that sentence. October 2010. Voice traffic in Skype is unaffected, although text messages are subject to DPI, and messages containing sensitive material, such as curse-words, are simply not delivered, with no notification provided to either participant in the conversation. China also blocks visual media sites like YouTube.com, and various photography and blogging sites.http://www.pcworld.com/article/id,138599-c,sites/article.html China Blocks YouTube, Restores Flickr and Blogspot PC World 2007-10-18 2008-03-03 Iran Internet censorship in Iran The Iranian government purchased a system, reportedly for deep packet inspection, in 2008 from Nokia Siemens Networks (NSN), a joint venture Siemens AG, the German conglomerate, and Nokia Corp., the Finnish cellphone company, according to a report in the Wall Street Journal in June, 2009, quoting NSN spokesperson Ben Roome. According to unnamed experts cited in the article, the system "enables authorities to not only block communication but to monitor it to gather information about individuals, as well as alter it for disinformation purposes." The system was purchased by the Telecommunication Infrastructure Co., part of the Iranian government's telecom monopoly. According to the Journal, NSN "provided equipment to Iran last year under the internationally recognized concept of 'lawful intercept,' said Mr. Roome. That relates to intercepting data for the purposes of combating terrorism, child pornography, drug trafficking and other criminal activities carried out online, a capability that most if not all telecom companies have, he said.... The monitoring center that Nokia Siemens Networks sold to Iran was described in a company brochure as allowing 'the monitoring and interception of all types of voice and data communication on all networks.' The joint venture exited the business that included the monitoring equipment, what it called 'intelligence solutions,' at the end of March, by selling it to Perusa Partners Fund 1 LP, a Munich-based investment firm, Mr. Roome said. He said the company determined it was no longer part of its core business." The NSN system followed on purchases by Iran from Secure Computing Corp. earlier in the decade."Iran's Web Spying Aided By Western Technology" by Christopher Rhoads in New York and Loretta Chao in Beijing, The Wall Street Journal, June 22, 2009. Retrieved 6/22/09. Questions have been raised about the reporting reliability of the Journal report by David Isenberg, an independent Washington, D.C.-based analyst and Cato Institute Adjunct Scholar, specifically saying that Mr. Roome is denying the quotes attributed to him and that he, Isenberg, had similar complaints with one of the same Journal reporters himself in an earlier story."Questions about WSJ story on Net Management in Iran" by David S. Isenberg, isen.blog, June 23, 2009. Retrieved 6/22/09. NSN has issued the following denial: NSN "has not provided any deep packet inspection, web censorship or Internet filtering capability to Iran.""Provision of Lawful Intercept capability in Iran" Company press release. June 22, 2009. Retrieved 6/22/09. A concurrent article in The New York Times said the NSN sale had been covered in a "spate of news reports in April [2009], including The Washington Times," and reviewed censorship of the Internet and other media in the country, but did not mention DPI."Web Pries Lid of Iranian Censorship" by Brian Stelter and Brad Stone, The New York Times, June 22, 2009. Retrieved 6/23/09. DPI and net neutrality network neutrality People and organizations concerned about privacy or network neutrality find inspection of the content layers of the Internet protocol to be offensive, saying for example, "the 'Net was built on open access and non-discrimination of packets!"http://www.cybertelecom.org/ci/neutral.htm#his Network Neutrality: Historic Neutrality Cybertelecom Genny Pershing 2008-06-26 Critics of network neutrality rules, meanwhile, call them "a solution in search of a problem" and say that net neutrality rules would reduce incentives to upgrade networks and launch next-generation network services.http://www.cybertelecom.org/ci/neutral.htm#ins Network Neutrality: Insufficient Harm Cybertelecom Genny Pershing 2008-06-26 Software OpendpiOpendpi is the open source version for non obfuscated protocols, PACE includes obfuscated/encrypted protocols like Skype or encrypted BitTorrent.Deep packet inspection engine goes open source L7-Filter is a classifier for Linux's Netfilter that identifies packets based on application layer data.L7-Filter home page It can classify packets such as Kazaa, HTTP, Jabber, Citrix, Bittorrent, FTP, Gnucleus, eDonkey2000 etc. It classifies streaming, mailing, P2P, VOIP, protocols and gaming applications. Hippie(Hi-Performance Protocol Identification Engine) is an open source project which was developed as kernel module.Hippie Project download page on SourceForge It was developed by Josh Ballard. It supports both DPI as well as firewall functionality. Hippie reference page SPID (Statistical Protocol IDentification) project is based on statistical analysis of network flows to identify application traffic.SPID project on SourceForge SPID algorithm can detect the application layer protocol (layer 7) by analysing flow (packet sizes etc.) and payload statistics (byte values etc.) from pcap files. It is just a proof of concept application and currently supports around 15 application/protocols like eDonkey Obfuscation traffic, Skype UDP and TCP, BitTorrent, IMAP, IRC, MSN etc. Tstat (TCP STatistic and Analysis Tool) provides insight into traffic patterns and gives details statistics for numerous applications and protocols.Tstat project home The open source community offers a wide array of options for performing deep packet inspection functions; a comprehensive list is maintained by the dPacket.org community https://www.dpacket.org/group-posts/open-source-software-general-discussion/open-source-software-related-deep-packet-inspect Open Source deep packet inspection projects dPacket.org See also Common carrier Deep content inspection Deep packet capture Firewall Foreign Intelligence Surveillance Act Golden Shield Intrusion prevention/detection systems Network neutrality NSA warrantless surveillance controversy Stateful firewall ECHELON References 2 External links Test Methodology - registration required Subverting Deep Packet Inspection the Right Way What is "Deep Inspection"? A collection of essays from industry experts What Is Deep Packet Inspection and Why the Controversy White Paper "Deep Packet Inspection – Technology, Applications & Net Neutrality" Egypt's cyber-crackdown aided by US Company - DPI technology used by Egyptian government in recent internet crackdown http://www.savetheinternet.com/ Deep Packet Inspection puts its stamp on an evolving Internet Validate DPI policy using real applications Category:Subject:Information technology Deep packet inspection es:Deep Packet Inspection fa:بازرسی ژرف بسته‌ها fr:Deep packet inspection ko:심층 패킷 검사 my:နက်ရှိုင်းသော အစုံလိုက်သုံးပက်ကက် စစ်ဆေးခြင်း nl:Deep packet inspection zh:深度分组检测