Toby Clarke (2009) Fuzzing for software vulnerability discovery.
Full text access: Open
Background Fuzz testing can be used to detect software programming flaws present in an application by submitting malformed input to the application as it executes. Some programming flaws impact upon the security of an application by undermining the performance of controls, rendering the application vulnerable to attack. Hence, the discovery of programming flaws can lead to the discovery of security vulnerabilities. Fuzz testing (like almost all run-time testing) does not require access to the source code, which makes it attractive to those who wish to assess the security of an application, but are unable to obtain access to the source code, such as end-users, corporate clients, security researchers and cyber criminals. Motivation The author wanted to explore the value of fuzz testing from the point of view of a corporate client that intends to release software including a component developed by a third party, where the component source code is not available for review. Three case studies where conducted: two practical fuzz testing methodologies ('blind' data mutation and protocol analysis-based fuzzing) were employed to discover vulnerabilities in a commercial operating system, and a purposefully vulnerable web server, respectively. A third case study involved the exploitation of a vulnerability discovered using fuzz testing, including the production of 'Proof of Concept' code. Conclusions It was found that fuzzing is a valid method for identifying programming flaws in software applications, but additional analysis is required to determine whether discovered flaws represented a security vulnerability. In order to better understand the analysis and ranking of errors discovered using fuzz testing, exploit code was developed based on a flaw discovered using fuzz testing. It was found that the level of skill required to create such an exploit depends (largely) upon the nature of the specific programming flaw. In the worst case (where user-controlled input values are passed to the instruction pointer register), the level of skill required to develop an exploit that permitted arbitrary code execution was minimal. Due to the scale and range of input data accepted by all but the most simple of applications, fuzzing is not a practical method for detecting all flaws present in an application. However, fuzzing should not be discounted since no current software security testing methodology is capable of discovering all present flaws, and fuzzing can offer benefits such as automation, scalability, and a low ratio of false-positives.
This is a Published version This version's date is: 17/02/2009 This item is peer reviewed
https://repository.royalholloway.ac.uk/items/4941b5d6-2f4a-8499-8954-1a7feee7cc4c/1/
Deposited by () on 23-Jun-2010 in Royal Holloway Research Online.Last modified on 15-Dec-2010
[1] Skillsoft 241292 eng - c sharp 2005: Serialization and i/o, Online Course Con-tent.
[2] What is a parser?, Online-Article, Site visited on 2nd August 2008, Not dated.,http://www.programmar.com/parser.htm.
[3] Dave Aitel, Spike.c, C source code, Part of the SPIKE Fuzzer Creation KitVersion 2.9, http://www.immunitysec.com/resources-freesoftware.shtml.
[4] , The advantages of block-based protocol analysis for security testing,http://www.net-security.org/dl/articles/advantages_of_block_based_analysis.pdf (2002).
[5] George A. Akerlof, The market for `lemons': Quality uncertainty and the marketmechanism, Quarterly Journal of Economics 84 (3): 488500., 1970, www.econ.ox.ac.uk/members/christopher.bowdler/akerlof.pdf.
[6] Pedram Amini and Aaron Portnoy, Primitives.py, part of the sulley fuzzingframework source code, Python source code, August 2007, http://www.fuzzing.org/2007/08/13/new-framework-release/.
[7] Danilo Bruschi, Emilia Rosti, and R. Ban , A tool for pro-active defense againstthe bu er overrun attack, ESORICS '98: Proceedings of the 5th European Sym-posium on Research in Computer Security (London, UK), Springer-Verlag, 1998,pp. 17{31.
[8] Jared DeMott, The evolving art of fuzzing, 2006, www.vdalabs.com/tools/The_Evolving_Art_of_Fuzzing.pdf, video.google.com/videoplay?docid=4641077524713609335.
[9] Mark Dowd, John McDonald, and Justin Schuh, The art of software securityassessment: Identifying and preventing software vulnerabilities, Addison-WesleyProfessional, 2006.
[10] Erwin Erkinger, Software reliability in the aerospace industry - how safe and re-liable software can be achieved, 23rd Chaos Communication Congress presenta-tion, 2006, http://events.ccc.de/congress/2006/Fahrplan/events/1627.en.html.
[11] Gadi Evron, Fuzzing in the corporate world, Presentation, 23rdChaos Communication Congress: Who can you trust?, December2006, http://events.ccc.de/congress/2006/Fahrplan/attachments/1248-FuzzingtheCorporateWorld.pdf.
[12] R. Fielding, J. Mogul, H. Frytsyk, L. Masinter, P. Leach, and T. Berners-Lee, Rfc2616 - hypertext transfer protocol-http/1.1, Technical Report, June 1999, IETF.
[13] James C. Foster and Vincent Liu, Writing security tools and exploits, SyngressPublishing, 2005.
[14] Andreas Fuschberger, Software Security course lecture, 2008, March.
[15] Lars Marius Garshol, Bnf and ebnf: What are they and how do they work?,Online-Article, http://www.garshol.priv.no/download/text/bnf.html.
[16] Patrice Godefroid, Random testing for security: blackbox vs. whitebox fuzzing,RT '07: Proceedings of the 2nd international workshop on Random testing (NewYork, NY, USA), ACM, 2007, pp. 1{1.
[17] Patrice Godefroid, Michael Y. Levin, and David Molnar, Automated whiteboxfuzz testing, Technical Report MS-TR-2007-58, May 2007, www.isoc.org/isoc/conferences/ndss/08/papers/10_automated_whitebox_fuzz.pdf.
[18] Dick Grune and Ceriel J. H. Jacobs, Parsing techniques: a practical guide, EllisHorwood, Upper Saddle River, NJ, USA, http://www.cs.vu.nl/~dick/PTAPG.html, 1990.
[19] Les Hatton, Five variations on the theme: Software failure: avoiding the avoid-able and living with the rest, University of Kent, Canterbury, November 2003.
[20] Greg Hoglund and Gary McGraw, Exploiting software: How to break code, Pear-son Higher Education, 2004.
[21] Michael Howard and Steve Lipner, The security development lifecycle, MicrosoftPress, Redmond, WA, USA, 2006.
[22] Senior Consultant Information Risk Management Plc, John Yeo, Personal con-versation, July 2008.
[23] Intel, Pentium processor family developer's manual, http://www.intel.com/design/pentium/MANUALS/24143004.pdf.
[24] Rauli Kaksonen, A functional method for assessing protocol implementation se-curity, VTT Publications, http://www.ee.oulu.fi/research/ouspg/protos/analysis/VTT2001-functional/, 2001.
[25] Rauli Kaksonen, M. Laakso, and A. Takanen, System security assessment throughspeci cation mutations and fault injection, Proceedings of the IFIP TC6/TC11
International Conference on Communications and Multimedia Security Issues ofthe New Century (Deventer, The Netherlands, The Netherlands), Kluwer, B.V.,2001, p. 27.
[26] Marko Laakso, Protos - mini-simulation method for robustness testing, Presen-tation, 2002, Oulu University Secure Programming Group.
[27] Scott Lambert, Fuzz testing at microsoft and the triage pro-cess, Online-Article, Site visited on 2nd August 2008, Septem-ber 2007, http://blogs.msdn.com/sdl/archive/2007/09/20/fuzz-testing-at-microsoft-and-the-triage-process.aspx.
[28] John Leyden, Penetrate and patch e-business security is grim, Online-Article, Published Wednesday 20th February 2002 09:42 GMT, Site visitedJuly 30th, 2008, http://www.theregister.co.uk/2002/02/20/penetrate_and_patch_ebusiness_security/.
[29] Tim Lindholm and Frank Yellin, Java virtual machine speci cation, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1999.
[30] B. Littlewood (ed.), Software reliability: achievement and assessment, BlackwellScienti c Publications, Ltd., Oxford, UK, UK, 1987.
[31] Thomas Maller, Rex Black, Sigrid Eldh, Dorothy Graham, Klaus Olsen, MaaretPyhjrvi, Geo Thompson, Erik van Veendendal, and Debra Friedenberg, Cer-ti ed tester | foundation level syllabus, Online-Article, April 2007, http://www.istqb.org/downloads/syllabi/SyllabusFoundation.pdf.
[32] Gary McGraw, Testing for security during development: Why we should scrappenetrate-and-patch, Online-Article, Site visited July 30th 2007, http://www.cigital.com/papers/download/compass-2.pdf.
[33] Microsoft, Info: Windows rundll and rundll32 interface, Online-Article, Novem-ber 2006, http://support.microsoft.com/kb/164787.
[34] Barton P. Miller, Louis Fredriksen, and Bryan So, An empirical study of thereliability of unix utilities, Commun. ACM 33 (1990), no. 12, 32{44.
[35] Charlie Miller, How smart is intelligent fuzzing - or - how stupid is dumb fuzzing?,Conference Presentation, Defcon 15, September 2007, http://video.google.co.uk/googleplayer.swf?docid=-6109656047520640962&hl=en&fs=true.
[36] Christiane Rutten, Fuzzy ways of nding aws, Online-Article, Site visited on1st August 2008, January 2008, http://www.heise-online.co.uk/security/Fuzzy-ways-of-finding-flaws--/features/100674.
[37] Joel Scambray, Mike Shema, and Caleb Sima, Hacking exposed web applications,second edition (hacking exposed), McGraw-Hill Osborne Media, 2006.
[38] Bruce Schneier, How security companies sucker us with lemons, Online-Article, April 2007, http://www.wired.com/politics/security/commentary/securitymatters/2007/04/securitymatters_0419.
[39] scut / team teso, Exploiting format string vulnerabilities, September 2001,crypto.stanford.edu/cs155/papers/formatstring-1.2.pdf.
[40] Beyond Security, Simple web server (sws) test case, Online-Article, http://www.beyondsecurity.com/sws_overview.html.
[41] S.E. Smith, What is garbage in garbage out?, Online-Article, Site visited July30th, 2007, http://www.wisegeek.com/what-is-garbage-in-garbage-out.htm.
[42] Sherri Sparks, Shawn Embleton, Ryan Cunningham, and Cli Zou, Automatedvulnerability analysis: Leveraging control ow for evolutionary input crafting,www.acsac.org/2007/papers/22.pdf.177
[43] , Sidewinder: An evolutionary guidance system for malicious input craft-ing, Presentation at Black Hat Conference, August 2006, www.blackhat.com/presentations/bh-usa-06/BH-US-06-Embleton.pdf.
[44] Brad Stone, Moscow company scrutinizes computer code for aws, Online-Article, January 2007, http://www.iht.com/articles/2007/01/29/business/bugs.php.
[45] Michael Sutton, Fuzzing - brute force vulnerability discovery, Presentation, RE-CON conference, Montreal, Canada, Friday June 16th 2006.
[46] Michael Sutton, Adam Greene, and Pedram Amini, Fuzzing: Brute force vulner-ability discovery, Addison-Wesley Professional, 2007.
[47] David Thiel, Exposing vulnerabilities in media software, Black Hat conferencepresentation, BlackHat EU 2008, www.isecpartners.com/files/iSEC_Thiel_Exposing_Vulnerabilities_Media_Software_bh07.pdf.
[48] Jacob West, How i learned to stop fuzzing and nd more bugs, Defcon conferencepresentation, available as a video recording, DefCon 15 Las Vegas, 2007, August2007, video.google.com/videoplay?docid=-5461817814037320478.
[49] Peter Winter-Smith and Chris Anley, An overview of vulnerability researchand exploitation, 2006, www.cl.cam.ac.uk/research/security/seminars/archive/slides/2006-05-16.ppt.