An Analysis on the Regulation of Grey Market Cyber Materials

Introduction

The creation of the computer network system and its spread throughout the international realm has opened doors for new ways of gathering information as well as manipulating this information for both protective and malicious purposes. While the material stored within computer systems is often thought of as being privy to the user of that system and whomever the user decides to share their data with, the rise in cyberattacks has proven otherwise. Cyber "crime" or "attacks"─the exploitation of Internet and, more generally, computer vulnerabilities in order to access and use private information ─rose as personal data began being stored on system networks since the nascent years of the internet.¹

Policies such as the Computer Fraud and Abuse Act of 1986² and the proposed National Defense Authorization Act for 2014³ have establish(ed) guidelines for unauthorized tapping into the systems of the U.S. government as well as financial and commerce institutions. While cyber-attacks were initially thought of as a method used by individual or small bands of hackers attempting to access information for personal use⁴, discussions of cyber-crime have shifted to the offensive mechanisms of hacking employed by governmental organizations. With large-scale attacks such as the Morris worm replicated in UNIX systems across the globe in 1988⁵ and, more recently, the infection of 30,000 Saudi Aramco (world's largest oil producer) networks with a self-replicating Shamoon virus in 2012,⁶ it has become clear that the title of "hacker" is no longer reserved for individuals but extends to governments as well.

The rise in government probes into both domestic and foreign security systems reveals that while there is a significant amount of information to be gathered for important national security purposes, there is also a significant level of risk in the methodology of information gathering.⁷ Risks are exacerbated by the types of markets available for purchasing exploits and botnets or merely the information on system vulnerabilities that can be crafted into an exploit.

The three forms of markets – the white market, grey market, and black market – all pose significant dangers in that they compromise the information of buyers, purchasers, and those who will be affected by the use of the cyber material being sold. This paper focuses on the clandestine and unregulated grey market where government authorities, defense organizations, and other pseudo-political groups are able to purchase their cyber material. Given the market's secrecy, this paper argues that regulatory measures ought to be imposed on grey market transactions.

More specifically, vendors should be required to disclose the vulnerabilities and exploits to computer software companies (in this essay, synonymous with the term "manufacturers") prior to making the information marketable to purchasers. Provisions for providing this information ought to be mandatory in an effort to ensure that companies have been given ample ability to secure their data and the data of their users. This suggestion is based on the notion that defensive measures of securing information ought to be prioritized over the potential to create offensive attacks using holes, or "vulnerabilities," in security.

The monetary incentive for hackers to sell cyber materials and information is related to the changing nature of the cyber realm

I begin by outlining the general functions of the grey market. Here, I explain the uses of the cyber materials─namely for offensive security tactics by government authorities and defense contractors. In describing the buyer-seller dynamic, I outline the effects of anonymity and monetary incentives on vendors within the market. I remark on the limitation these dynamics place on regulation.

Following this, I explore the possibility of regulation through a previously advocated technique, professionalization, and the mandatory disclosure. In exploring regulation, I discuss the types weapons sold in the market and types of buyers. I narrow my analysis to the type of cyber-material predominately bought and sold in the market: zero-day exploits. My explanation of zero-day exploits includes the risks associated with cyber materials being indiscriminately made available to all types of actors. Throughout my analysis, I consider concerns of feasibility. In discussing the merits of market regulation, I hope to demonstrate that regulation is essential to ensuring security for the systems vulnerabilities infect.

The Grey Market Allure

A description of the grey market yields three features key to its function as an unregulated space: the culture of anonymity, the potential for large monetary gains, and (most relevant to government and defense agencies) the potential for gaining information necessary for offensive attacks. Anonymity─the confidentiality of actors and specific materials on the grey market─is a particularly important issue to both sellers, and most especially buyers. Understandably, agencies prefer to keep their purchases unknown to the general public, as their activity within the grey markets is part of national security intelligence and defense informational tactics.⁸

For example, the Grugq ─an exploit broker─faced immense pressure and backlash in the form of decreased demand for his informational products after exposing in an interview⁹ the payments and customers he receives for cyber exploits. The nature of the market means that the threat of backlash and subsequently lost revenue undermines efforts to expose information and reinforces the culture of anonymity. Secondly, individual hackers who discover system holes─be it at hacking conferences amongst colleagues and potential buyers or within the privacy of their home─can make upwards of 15% from their sales.¹⁰ The monetary incentive for hackers to sell cyber materials and information is related to the changing nature of the cyber realm. Perlroth and Sanger note, "Ten years ago, hackers would hand knowledge of such flaws to Microsoft and Google free, in exchange for a T-shirt or perhaps for an honorable mention on a company's Web site".¹¹

As cyber security is becoming more and more part of the U.S.'s national security discourse¹², authorities are seeking assistance from those with intimate knowledge of the hacking community. As a result of the changing face of cyber attacks, hackers are now being sought out by reputable states agencies such as the National Security Agency¹³ who have become "open advocates [of the hacker community], willing to buy technology, and fund research",¹⁴ as their work is being demanded by the shifting nature of warfare and the intelligence agency. Initially, infiltrating the hacker community had its own challenges, particularly in the area of trustworthiness amongst hackers and governments. To mollify suspicions, transactions were based on a level of anonymity.

This was necessary for hackers who had previously been targeted by government agencies and for governments who did not yet have the intelligence and expertise to verify information retrieved from hacking communities. Currently, individuals who were once stigmatized as criminals are parts of a lucrative market that reinforces the specificity of their work. This also means agencies are willing to pay more in order to remain atop of the demands of the shifting cyber tools, thereby reinforcing the effects of anonymity and monetary incentives.

While supply and demand in competitive markets would predict that the lucrative market for hackers would create an opposing market for researchers seeking to patch loopholes, this is not currently the case. Despite the work that some researchers do to uncover vulnerabilities and forewarning software companies of future breaches, these individuals are oftentimes not compensated for their work─even by large corporations such as Adobe and Apple¹⁵.

In fact, independent hackers who sell their findings to grey market vendors make close to "10 or even 100 times" more than what researchers can receive for providing information to companies such as Facebook and Microsoft¹⁶, thereby creating an indisputable disincentive to assist companies. Efforts such as the "Hacker One" initiative by Facebook and Microsoft as well as the Texas─based Zero Day Initiative¹⁷ have attempted to amend the problems of compensation for disclosure by paying researchers $300 to $5,000¹⁸ for their knowledge of any malware, vulnerabilities, or exploits that will later be sold.

On Hacker One, discussions of vulnerabilities or "bugs" are made open on feeds that also list who have earned certain compensations or "bounties" for their contributions.¹⁹ Hackers can even ban together forming "teams" for exploring holes in systems and reporting back to Hacker One.²⁰ Yet, it is clear that any effort to require disclosure by hackers to companies would have to offer comparable monetary incentives to promote cooperation and avoid the risk of merely driving transactions even further underground into deeper spaces of anonymity such as the black market for cyber materials. Hence, international regulation within the hacker and cyber researcher communities is limited.

Mandatory Disclosure as Regulation: A Possibility?

Within the grey market is a culture of indiscriminate transactional practices, which pose difficulties for requiring disclosure, beyond concerns of monetary incentives. Buyers and sellers that are not held accountable by outside standards are susceptible to equipping organizations with malicious intent with the products they need to carry out their actions. One suggestion to rectify this risk is to make the grey market sellers "official" by professionalizing their practices. Although this suggestion is more focused on changing the perception or connotation associated with the clandestine sale of cyber materials, professionalization would involve setting normalized standards for the industry. The goal of professionalization would be to "regulate workers…enhance public trust…and enable compliance with regulatory requirement" among other goals.²¹

Presumably, this would involve standardizing requirements on the levels of schooling needed to enter the cyber security field, assessment of skills related to the industry and general career preparation/ etiquette when handling personable, vulnerable information. Not only would standardizing the players of the market entail requirements for credentials such as certifications for hackers and required training for properly handling sensitive information, but it would also affect the way vendors presently operate by not screening their buyers. The wellknown French organization Vupen, which specializes in identifying vulnerabilities to be sold for large sums (sometimes $1 million),²² refuse to reveal their clientele.²³ By not screening buyers, Vupen and companies like it create the potential for maliciously intended agencies to gain access to their vulnerabilities without any pushback from regulatory practices.

As a result of the changing face of cyber attacks, hackers are now being sought out by reputable states agencies such as the national security agency

Professionalization seems to be a more attractive option to naturally regulate grey markets because it involves altering the culture of the market. Yet, determining hiring practices, specifying types of skill sets, and giving guidelines for the type of education necessary to enter into these markets as a vendor or identifier of vulnerability are difficult tasks. In this way, the capricious and blanketed nature of discovering vulnerabilities and creating exploits hinders professionalization. Hence, what works in one instance of uncovering and creating a potential exploit to be sold may not be sufficient or necessary to effectively identify holes in another scenario. The consistency of professionalized standards is, in fact inconsistent with the market itself.

Another suggestion, the one advocated in this paper, is to make transactions a liability for the seller.²⁴ If the seller is required to demonstrate proof (either via documentation or exchanges between the vendor and company) that the vulnerabilities and exploits being sold have also been given to the manufactures of the systems that the holes are within, then the risk of those vulnerabilities being given to anonymous buyers is mitigated, if not eliminated.

The most salient issue surrounding anonymity, as mentioned before, is that the intention of the buyer is unknown. In the case where a company has been made aware that their systems could be compromised, the company could now be held liable for breaches and have the responsibility of pursuing measures to mend the breach. Intentions of buyers are then made irrelevant, since, regardless of who knows the vulnerability, the company is well informed on how to protect themselves and their users.

While the approach of liability is most beneficial in cases where the intentions are unclear, the viability of requiring disclosure of information from sellers of vulnerabilities is complicated by situations where the vulnerabilities can be used for preventative (offensive) purposes. Perhaps the most salient issue related to feasibility is the need to preserve avenues for cyber information gathering to be used in offensive activities. Government and defense agencies typically purchase cyber materials as tools for espionage and claim these practices as legitimate forms of warfare and, more benignly, national security.

The offensive nature of espionage relies on the fact that the exploit is used to obtain information to help promote other security initiatives. Claims to the necessity of offensive attacks are difficult to counteract since the idea of offensive tools is not foreign to the realm of warfare. The case of Stuxnet illuminates a concern in providing governments with access to vulnerabilities on a private marketplace. Stuxnet, a zeroday weaponized computer program discovered in June 2010 by VirusBlokAda, was developed from codes exposing vulnerabilities in Windows systems.²⁵ The United States and Israel were later revealed as being the initiators of the cyber-attack, with 60,000 computers²⁶ in various countries such as the United Kingdom, Germany, South Korea, China, and India being infected and over 60% of the worm infections taking place in Iran.²⁷

The sheer scale and global nature of the attack was not the sole reason for concern. Stuxnet disrupted the frequencies within centrifuges of Iran's nuclear facilities, thereby becoming the "first industrial-sized" attack using malware. The intention was to isolate systems within Iran; however, the fact that 40% of the worm was replicated in other nations exposed risks related to collateral damage.

The concern, related to the idea of intentionality and anonymity, is that two subsets of users of the grey market─terrorist organizations and "rogue" governments – may have (or already do have) the same information as more favorable (or trustworthy) buyers.

Furthermore, regulation, as during prohibition, can cause otherwise visibly detrimental actions to become less visible and transactions to occur in the "underground" black market. Having exploits and vulnerabilities in that market is potentially even more risky since knowledge of where these markets are and who is partaking in transactions is kept secretive, in addition to what is being sold. And, even more starkly different from the grey market, black-market cyber materials (oftentimes counterfeit)²⁸ are sold at cheap prices to attract purchasers who would be more inclined to buy from more secure vendors at the same price. Furthermore, materials are oftentimes counterfeit.

The Trajectory of Cyber Warfare and Perception of Risks

Thus far, the focus of this paper has been on the actors of the grey market and the risks associated with their participation. The question of the necessity of regulation cannot focus solely on concerns of who is selling and who is purchasing. Rather, there is also a question to be considered about what is being sold; and the main product on the line is one that has been linked to growing discussions on the future of cyber warfare: exploits.

Zeroday exploits are known as the most detrimental compromises of software because they are classified as "unknown threats". Thereby, zerodays cannot be mended by anti-spyware or malware protections already in place within a system.²⁹ Sold on the "high-end" level of the gray market at "up to $250,000,"³⁰ once a zero-day has infiltrated a system, the effects are immediate and system managers do not have time ("zero days") to reverse and patch the holes within the system.³¹

Zero-days are considered to be "weaponized" once they are used "to disrupt, disable, or destroy computer networks and their components."³² While zero-days are the most prevalent types of exploits to be weaponized, the information needed to transform a system into a botnet can also be found in the grey market. Specifically, a botnet is when hackers use malware to control systems remotely, thereby being able to access private documents and information stored on systems, as well as control commands with the computer(s).³³ These controls can be used for espionage or fraudulent purposes.

The question of the necessity of regulation cannot focus solely on concerns of who is selling and who is purchasing

Cyber war has become more of a catch phrase because of the prevalence of interconnected informational and communications technologies; yet, the perception of risk is a limiting factor in how willing individuals are or will be to abide by disclosure requirements. Currently, many still believe that cyber war is not imminent and that the manipulation of systems technology is confined to the realm of personal theft, such as identity thieves or clandestine government activity that ought to remain privy to the Department of Defense and National Security Association.

Yet, the real concern surrounding cyber materials purchased in the grey market by ill-intended actors lies in the ability for information and communications technologies now to be used for targeting "electrical grids, food distribution systems, [and] any essential infrastructure that runs on computers."³⁴ Perceptions of risk matter in so far as hackers are individual members of these communities. They are also the ones who can create a culture within the hacking industry of either ambivalence towards the growing potential to cause physical damage or activism for safer methods of utilizing information for the sake of all those involved.

Companies who sell informational and technological material face enormous costs once their information is compromised. The International Criminal Police Organization (INTERPOL) acknowledges these large costs in their 2007 and 2008 estimates of sums paid to retroactively rectify attacks against gaps in software. INTERPOL notes, "The cost of cybercrime worldwide was estimated at approximately USD 8 billion. As for corporate cyber espionage, cyber criminals have stolen intellectual property from businesses worldwide worth up to USD 1 trillion" ─ larger than the budget of INTERPOL itself.³⁵ Similarly, Intel reports that the "US government alone spent $25 million purchasing code vulnerabilities"³⁶ making it the largest spender on material for potentially offensive exploits in comparison to states such as Russia, China, and North Korea.

While the U.S. government justifies their purchases through claims of using information for offensive attacks, it is unclear whether the same discretion can be trusted in the hands of other actors within the grey market, as mentioned above. Terrorist organizations are also able to purchase exploits and vulnerabilities, and the absence of external checks on their purchases means that they are unregulated in the most extreme sense and have the ability to weaponize vulnerabilities. Essentially, mandatory disclosure has the potential to minimize these costs by creating a safety net for software companies who have the human capital to invest in finding solutions to patch the holes in their software, but do not necessarily have information on where these holes are located.

Conclusion

At the beginning of this paper, I hoped to demonstrate the necessity for regulating the grey market of cyber materials. The value in regulating the grey market was based in the idea that informational security ought to be prioritized over offensive security measures that could be pursued by access to system vulnerabilities. Throughout this paper, it has become clear that these two intentions ─ securing information and pursuing offensive attacks ─ are not mutually exclusive. With cases such as Stuxnet, government agencies have used the vulnerabilities and zero-day exploits purchased from grey market vendors as a means of preventative action. Yet, the feasibility of requiring sellers to disclose information of their "products" is limited by the attractiveness of lucrative jobs in hacking as well as the willingness of buyers and sellers in the grey market to impose regulations on their practices, as demonstrated by the stance of organizations like Vupen.

Despite these limitations, it is necessary to consider mandatory disclosure as an option because of the real risks associated with maintaining a secretive market for cyber materials. These risks are amplified by the types of materials primarily sold in the grey market (zero-day exploits) and the types of buyers attracted to the market's clandestine nature (organizations with potentially malicious intent). While government agencies argue the necessity of the grey market for purchasing offensive warfare material, the necessity to protect individuals ought to be valued over protecting an interest in maintaining a wide array of national security tools. Requiring the disclosure of vulnerabilities would enable companies to pursue security measures while not fully hindering buyers from purchase exploits and vulnerabilities on the grey market.

References

Bronk, Christopher and Eneken Tikk-Ringas, “The Cyber Attack on Saudi Aramco.” Survival: Global Politics and Strategy. Vol 55 no.2 April-May 2013, Pg 81-82. <http://www.tandfonline.com/doi/pdf/10.1080/00396338.2013.784468>

Farwell, James P. and Rafal Rohozinski, “Stuxnet and the Future of Cyber War.” Survival: Global Politics and Strategy. Vol 53 no.1, February-March 2011.Routledge. Pg 23-30. <http://www.tandfonline.com/doi/pdf/10.1080/00396338.2011.555586>

Goldsmith, Jack. “Regulation of the Internet: Three Persistent Fallacies.” Chicago-Kent Law Review, Vol 73, Issue 4, Symposium on the Internet and Legal Theory. 11 Oct. 2013. Web. 1 March. 2014. <http://scholarship.kentlaw.iit.edu/cgi/viewcontent.cgi?article=3143&context=cklawreview>

Greenberg, Andy, “Meet the Hackers Who Sell Spies the Tools to Crack Your PC (And Get Paid Six Figure Fees),” Forbers, March 12, 2012, <http://www.forbes.com/sites/andygreenberg/2012/03/21/meet-the-hackers-who-sell-spies-the-tools-to-crack-your-pc-and-get-paid-six-figure-fees/>

H.R. 1960: National Defense Authorization Act for Fiscal Year 2014, 113th Congress, 2013-2015. Text as of Jul 08, 2013. <https://www.govtrack.us/congress/bills/113/hr1960/text> [Internet accessed on March 2nd 2014]

HackerOne <https://hackerone.com/feed> [Internet Accessed on February 27, 2014] International Criminal Police Organization. <http://www.interpol.int/Crime-areas/Cybercrime/Cybercrime> [Internet accessed on March 1st 2014]

Langer, Ralph, “Stuxnet: Dissecting a Cyberwarfare Weapon.” IEEEXplore, Focus. May/June 2011. Web. 1 March 2014. <http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5772960&tag=1>

Menn, Joseph, “Special Report: U.S. Cyberwar Strategy Stokes Fear of Blowback,” Reuters, May 10, 2013, <http://in.reuters.com/article/2013/05/10/usa-cyberweapons-idINDEE9490AX-20130510?type=economicNews>

Microsoft Safety and Security Center, “What is a botnet?,” http://www.microsoft.com/security/resources/botnet-whatis.asp.x[Internet accessed on February 27, 2014]

National Academies of Sciences, “Professionalizing the Nation’s Cybersecurity Workforce?: Criteria for Decision-Making.” The National Academies Press. (2013).Web. 7 Feb. 2014. Pg 16-18. <https://classesv2.yale.edu/access/content/group/plsc125_s14/Readings%20-%20Week%205/2013%20-%20Professionalising%20the%20Nations%20Cybersecurity%20Workforce.pdf>

Nato Review Magazine, <http://www.nato.int/docu/review/2013/Cyber/timeline/EN/index.htm> [Internet accessed on February 24, 2014]

PcTools by Symantec, “What is a Zero-Day Vulnerability?,” http://www.pctools.com/securitynews/zero-day-vulnerability/ [Internet Accessed on February 27, 2014]

Pereloth, Nicole and David E. Sanger, “Nations Buying as Hackers Sell Flaws in Computer Code,” 13 July 2013, The New York Times, <http://www.nytimes.com/2013/07/14/world/europe/nations-buying-as-hackers-sell-computer-flaws.html?_r=0>

Riley, Michael and Ashlee Vance. “Cyber Weapons: The New Arms Race,” Bloomberg BusinessWeek, 20 July 2011. Web. 3 March 2014. <http://www.businessweek.com/printer/articles/540-cyber-weapons-the-new-arms-race>

Rosenquist, Matthew, “How Offensive Cybersecurity is Changing the Industry,” Intel, 8 Oct 2013. Web. 1 March 2014. <https://communities.intel.com/community/itpeernetwork/blog/2013/10/08/how-offensive-cyber-security-is-changing-the-industry>

Stockton, Paul and Michele Golabek-Goldman. “Curbing the Market for Cyber Weapons.” (2013). Pg 2-12. <https://classesv2.yale.edu/access/content/group/plsc125_s14/Readings%20-%20Week%205/2013%20-%20Golabek-Goldman%20_%20Stockton%20-%20Curbing%20the%20Market%20for%20Cyber%20Weapons.pdf>

The White House, “Launch of the Cybersecurity Framework,” <http://www.whitehouse.gov/thepress-office/2014/02/12/launch-cybersecurity-framework>12 Feb. 2014. [Internet Accessed 17 Feb 2014]

U.S. Code § 1030 - Fraud and related activity in connection with computers. <http://www.law.cornell.edu/uscode/text/18/1030> [Internet accessed on March 2nd 2014]

Verizon, “2013 Data Breach Investigations Report,” Pg. 6, <file:///Users/kelseyannu-essuman/Downloads/rp_data-breach-investigations-report-2013_en_xg.pdf>. 2013. Web. 2 March 2014.

Zero Day Initiative <http://www.zerodayinitiative.com/about/> [Internet Accessed on March 2nd 2014]

Endnotes

1. Interpol, <http://www.interpol.int/Crime-areas/Cybercrime/Cybercrime>
2. 18 U.S. Code 1030 Fraud and Related Activity in Connection with Computers, <http://www.law.cornell.edu/uscode/text/18/1030>
3. Subtitle D: Cyber-Related Matters, National Defense Authorization Act for Fiscal Year 2014, <https://www.govtrack.us/congress/bills/113/hr1960/text>
4. See note 1
5. Cyber Timeline, <http://www.nato.int/docu/review/2013/Cyber/timeline/EN/index.htm>
6. Page 81-83, The Cyber Attack on Saudi Aramco, <http://www.tandfonline.com/doi/pdf/10.1080/00396338.2013.784468>
7. Pg 5-6, 2013 Data Breach and Investigation Report, <file:///Users/kelseyannu-essuman/Downloads/rp_data-breach-investigations-report-2013_en_xg.pdf>
8. Perlroth and Sanger, “Nations Buying as Hackers Sells Flaws in Security Code,” <http://www.nytimes.com/2013/07/14/world/europe/nations-buying-as-hackers-sell-computerflaws.html?_r=0>
9. Greenderg, Andy,”Meet the Hackers Who Sell Spies the Tools to Crack Your PC” <http://www.forbes.com/sites/andygreenberg/2012/03/21/meet-the-hackers-who-sell-spies-thetools-to-crack-your-pc-and-get-paid-six-figure-fees/>
10. Ibid
11. See note 8
12. The White House, Launch of the Cybersecurity Framework, http://www.whitehouse.gov/thepress-office/2014/02/12/launch-cybersecurity-framework
13. See note 8
14. Rosenquist, “How Offensive Cyber Security is Changing the Industry” <”https://communities.intel.com/community/itpeernetwork/blog/2013/10/08/how-offensive-cyber-security-is-changing-the-industry>
15. Menn, “U.S. Cyber strategy stokes fear of blowback,” <http://in.reuters.com/article/2013/05/10/usa-cyberweapons-idINDEE9490AX20130510?type=economicNews>
16. Stockton and Golabek-Goldman. “Curbing the Market for Cyber Weapons,” Pg 2-12 <https://classesv2.yale.edu/access/content/group/plsc125_s14/Readings%2-%2Week%25/2013%20-%20Golabek-Goldman%20_%20Stockton%20-%20Curbing%20the%20Market%20for%20Cyber%20Weapons.pdf>
17. Zero Day Initiative, <http://www.zerodayinitiative.com/about/>
18. See note 16
19. Hacker One, <https://hackerone.com/feed>
20. Ibid
21. Pg 16-18, National Academies of Sciences, “Professionalizing the Nation’s Cybersecurity Workforce?: Criteria for Decision-Making,” <https://classesv2.yale.edu/access/content/groupplsc125_s14/Readings%20-%20Week%205/2013%20-%20Professionalising%20the%20Nations%20Cybersecurity%20Workforce.pdf>
22. See note 9
23. Ibid
24. See note 21
25. Langer, <http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5772960>
26. Farewell and Rohozinski, Pg 23-30, "Stuxnet and the future of Cyber War" <http://www.tandfonline.com/doi/pdf/10.1080/00396338.2011.555586>
27. Ibid
28. See note 1
29. PcTools by Symantec, “What is a Zero-Day Vulnerability” <http://www.pctools.com/security-news/zero-day-vulnerability/>
30. Riley and Vance, “Cyber Weapons: The New Arms Race,” <http://www.businessweek.com/printer/articles/540-cyber-weapons-the-new-arms-race>
31. See note 29
32. See note 16
33. See note 30
34. Ibid
35. See note 1
36. Rosenquist, “How Offensive Cyber Security is Changing the Industry” <https://communities.intel.com/community/itpeernetwork/blog/2013/10/08/how-offensive-cyber-security-is-changing-the-industry>

Image Attributions

“Cyber Security at the Ministry of Defence MOD” by Harland Quarrington. Licensed under the Open Government Licence v1.0 via Wikimedia Commons.

“Computer-Police” by Stijn.Berghmans-Own Work. Licensed under the Creative Commons CC0 1.0 Universal Public Domain Dedication via Wikimedia Commons.

“Defense.gov Photo Essay” by U.S. Air Force Master Sgt. Jerry Morrison. Work of employee or official in U.S. Federal Government, permission in public domain via Wikimedia Commons.