Archive for the ‘Uncategorized’ Category

On Pentesting, Professionalism, & “Chill”

Tuesday, September 13th, 2016

After a recent penetration test report-out call with a client, I asked my interns if anything from the call surprised them. ChillOne of them noted that he was surprised how “chill” the call was. That was interesting to me because it reminded me that I had thought the exact same thing when I first got into consulting and pentesting. It’s easy to see how a readout call could be an incredibly tense, combative affair but in my experience the best pentesters manage to not only avoid that but reverse it.

The mood of the report-out call is an excellent barometer for something that’s critical, and often lacking, in our industry: a constructive relationship between the red team and blue team. While critical, it’s also subtle, and creating the conditions for a good relationship is a process that requires real work and empathy for everyone at the table. My advice: Start Early, Be Meticulously Professional, and Remember the Goal.

Start Early

If the goal is a relaxed, productive, (even “fun”) readout call, then the groundwork must start early. While there are other things that come before it, a detailed kickoff is really the first big chance to get moving in the right direction. As a tester your goal should be to make sure that everyone is clear and comfortable with what’s about to happen, and what exactly the client hopes to get out of it. The behavior ends up being a combination of a lot of standard questions, and sniffing around for any hint that there are either concerns or complexities going unaddressed. It’s also important at this point to really understand context from the client’s view. What’s a critical vs. a high or medium? What do they care less about than you might expect them to? Why? The more understanding you have now, the more that the entire report can be placed in context. If it feels like you’re facilitating a group therapy session where the clients are sharing their (security-relevant) hopes and their fears, then you’re probably doing something right. My team literally asks questions like “what keeps you up at night?” and “what’s the scariest thing we could do here?” Asking the big questions frankly and early helps take the elephant out of the room and moves toward productive discussion of the big questions rather than tentatively working up to them through peripheral issues.

Aside from the kickoff meeting itself, “start early” means start doing things well now so that you have a buffer of goodwill to draw on later. I’ve heard it called an “emotional bank account”: make people feel good about you, make a deposit; let them down, make a withdrawal. Ideally you always want that balance going up, but when something happens (and it’s definitely going to), you want to make sure that you’ve got a nice buffer of goodwill so that it’s understood that it was a blip in an otherwise solid relationship. Neil Gaiman once explained that people keep working because their work is good, they’re pleasant to work with, and because they deliver on time. But the secret, he says, is that it only takes two out of the three. Different people are going to be able to take a different two for granted, but know that if you always shoot for all three you’ve bought yourself some leeway if something happens.

Be Meticulously Professional

Beyond simply good will, one of the important reasons clients call in pentesters (or consultants of any kind) is to get that feeling that they’re in good hands; that someone is going to make sure that messy, complex things get taken care of properly. We’re expected to drop into situations where deadlines, resources, or nerves are already in trouble and provide some useful answers and confidence that the “Right Things” are being done. So, standard consulting practices like “Communicate well and often”, and “Don’t surprise people” apply, of course.  

But one area that security folks sometimes struggle with is ego. There typically are already plenty of personalities and internal politics involved; that makes it critical for us as outsiders to not bring further ego into the situation.

This ego can take a few forms. The first is a tendency toward fearmongering and overselling findings; wanting to be perceived as one of those “scary hacker types”. That can be helpful (to a point) for establishing technical credibility but it’s important to realize that being cool isn’t in the job description. Likewise, neither is taking credit or passing blame. Remember: Amateurs get credit, professionals get paid. The rule for blame is similar: as in the airport in Fight Club, never imply ownership of the bug. If the goal is to make something more secure, it’s rarely relevant who exactly created a bug when it’s likely process, tooling, or training that really needs to change.   

Remember The Goal

This leads into another place that unhelpful ego pops up: security absolutism.

I hear security absolutism in language like “Windows sucks because…” or “Well, actually there’s no point in fixing that because hackers could still…” (or really anytime someone starts handwaving about esoteric TLS attacks or Van Eck phreaking … you know the type).  

Real professionals need to be able to set aside the hacker mindset long enough to have productive, nuanced discussions about how to fix things. There are rarely perfect solutions, and the imperfect ones come with tradeoffs. We should all be willing to be as pragmatic on defense as we are on offense. The perception that security people are going to naysay or ridicule every suggestion hurts all of us, and makes us less effective as an industry. The “Nick Burns” mentality is a self-reinforcing stereotype we need to fight against at each encounter. Similarly, there’s an odor of superiority that often comes off some pentesters when they break a thing and speak about it publicly, as if that somehow demonstrates that they are smarter than the person who designed it. Sometimes a thing is *so* bad that an example must be made, but for my tastes those instances are far more rare than twitter and blogs would make you think.    

Even if a client is a pain to work with, doesn’t take good advice, and fights you on everything, they made at least one smart call: they asked for help. The better we’re able to appreciate that, understand their perspective, and work toward improving the system, the better the relationship and better the results. I’ll feel happier about our industry when dev and ops actually look forward to their calls with security folks, and I’ll tell you this: life’s a lot better when we look forward to them too. So remember; be professional, be empathetic, be helpful — and be chill.

Burp Extensions in Python and Pentesting Custom Web Services

Monday, September 16th, 2013

A lot of my work lately has involved assessing web services; some are relatively straightforward REST and SOAP type services, but some of the most interesting and challenging involve varying degrees of additional requirements on top of a vanilla protocol, or entirely proprietary text-based protocols on top of HTTP. Almost without fail, the services require some extra twist in order to interact with them; specially crafted headers, message signing (such as HMAC or AES), message IDs or timestamps, or custom nonces with each request.

These kind of unusual, one-off requirements can really take a chunk out of assessment time. Either the assessor spends a lot of time manually crafting or tampering with requests using existing tools, or spends a lot of time implementing and debugging code to talk to the service, then largely throws it away after the assessment. Neither is very good use of time.

Ideally, we’d like to write the least amount of new code in order to get our existing tools to work with the new service. This is where writing extensions to our preferred tools becomes massively helpful: a very small amount of our own code handles the unusual aspects of the service, and then we’re able to take advantage of all the nice features of the tools we’re used to as well as work almost as quickly as we would on a service that didn’t have the extra proprietary twists.

Getting Started With Burp Extensions

Burp is the de facto standard for professional web app assessments and with the new extension API (released December 2012 in r1.5.01) a lot of complexity in creating Burp extensions went away. Before that the official API was quite limited and several different extension-building-extensions had stepped in to try to fill the gap (such as Hiccup, jython-burp-api, and Buby); each individually was useful, but collectively it all resulted in confusing and contradictory instructions to anyone getting started. Fortunately, the new extension API is good enough that developing directly against it (rather than some intermediate extension) is the way to go.

The official API supports Java, Python, and Ruby equally well. Given the choice I’ll take Python any day, so these instructions will be most applicable to the parseltongues.  Getting set up to use or develop extensions is reasonably straightforward (the official Burp instructions do a pretty good job), but there are a few gotchas I’ll try to point out along the way.

  1. Make sure you have a recent version of Burp (at least 1.5.01, but preferably 1.5.04 or later where some of the early bugs were worked out of the extensions support), and a recent version of Java
  2. Download the latest Jython standalone jar. The filename will be something like “jython-standalone-2.7-b1.jar” (Event though the 2.7 branch is in beta I found it plenty stable for my use; make sure to get it so that you can use Python 2.7 features in your extensions.)
  3. In Burp, switch to the Extender tab, then the Options sub-tab. Now, configure the location of the jython jar.

    configurejython
  4. Burp indicates that it’s optional, but go ahead and set the “Folder for loading modules” to your python site-packages directory; that way you’ll be able to make use of any system wide modules in any of your custom extensions (requests, passlib, etc). (NOTE: Some Burp extensions expect that this path will be set to the their own module directory. If you encounter errors like “ImportError: No module named Foo”, simply change the folder for loading modules to point to wherever those modules exist for the extension.)
  5. The official Burp docs include one other important step:

    Note: Because of the way in which Jython and JRuby dynamically generate Java classes, you may encounter memory problems if you load several different Python or Ruby extensions, or if you unload and reload an extension multiple times. If this happens, you will see an error like:

    java.lang.OutOfMemoryError: PermGen space

    You can avoid this problem by configuring Java to allocate more PermGen storage, by adding a -XX:MaxPermSize option to the command line when starting Burp. For example:

    java -XX:MaxPermSize=1G -jar burp.jar
  6. At this point the environment is configured; now it’s time to load an extension. The default one in the official Burp example does nothing (it defines just enough of the interface to load successfully), so we’ll go one step further. Since several of our assessments lately have involved adding some custom header or POST body element (usually for authentication or signing), that seems like a useful direction for a “Hello World”. Here is a simple extension that inserts data (in this case, a timestamp) as a new header field and at the end of the body (as a Gist for formatting). Save it somewhere on disk.

    # These are java classes, being imported using python syntax (Jython magic)
    from burp import IBurpExtender
    from burp import IHttpListener

    
    

    # These are plain old python modules, from the standard library # (or from the “Folder for loading modules” in Burp>Extender>Options) from datetime import datetime

    
    

    class BurpExtender(IBurpExtender, IHttpListener):

    
    

    def registerExtenderCallbacks(self, callbacks):         self._callbacks = callbacks         self._helpers = callbacks.getHelpers()         callbacks.setExtensionName(“Burp Plugin Python Demo”)         callbacks.registerHttpListener(self)         return

    
    

    def processHttpMessage(self, toolFlag, messageIsRequest, currentRequest):
    # only process requests
    if not messageIsRequest:
    return

    
    

    requestInfo = self._helpers.analyzeRequest(currentRequest)         timestamp = datetime.now()                print “Intercepting message at:”, timestamp.isoformat()

    
    

    headers = requestInfo.getHeaders()
    newHeaders = list(headers) #it’s a Java arraylist; get a python list
    newHeaders.append(“Timestamp: ” + timestamp.isoformat())

    
    

    bodyBytes = currentRequest.getRequest()[requestInfo.getBodyOffset():]         bodyStr = self._helpers.bytesToString(bodyBytes)         newMsgBody = bodyStr + timestamp.isoformat()

    
    

    newMessage = self._helpers.buildHttpMessage(newHeaders, newMsgBody)
    print “Sending modified message:”
    print “———————————————-”
    print self._helpers.bytesToString(newMessage)
    print “———————————————-\n\n”
    currentRequest.setRequest(newMessage)
    return

  7. To load it into Burp, open the Extender tab, then the Extensions sub-tab. Click “Add”, and then provide the path to where you downloaded it.
  8. Test it out! Any requests sent from Burp (including Repeater, Intruder, etc) will be modified by the extension. Output is directed to the tabs in the Extender>Extensions view.
A request has been processed and modified by the extension. Since Burp doesn't currently have any way to display what a response looks like after it was edited by an extension, it usually makes sense to output the results to the extension's tab.

A request has been processed and modified by the extension. Since Burp doesn’t currently have any way to display what a response looks like after it was edited by an extension, it usually makes sense to output the results to the extension’s tab.

This is a reasonable starting place for developing your own extensions. From here it should be easy to play around with modifying the requests however you like: add or remove headers, parse or modify XML or JSON in the body, etc.

It’s important to remember as you’re developing custom extensions that you’re writing against a Java API. Keep the official Burp API docs handy, and be aware of when you’re manipulating objects from the Java side using Python code. Java to Python coercions in Jython are pretty sensible, but occasionally you run into something unexpected. It sometimes helps to manually take just the member data you need from complex Java objects, rather than figuring out how to pass the whole thing around other python code.

To reload the code and try out changes, simply untick then re-tick the “Loaded” checkbox next to the name of the extension in the Extensions sub-tab (or CTRL-click).

Jython Interactive Console and Developing Custom Extensions

Between the statically-typed Java API and playing around with code in a regular interactive Python session, it’s pretty quick to get most of a custom extension hacked together. However, when something goes wrong, it can be very annoying to not be able to drop into an interactive session and manipulate the actual Burp objects that are causing your extension to bomb.

Fortunately, Marcin Wielgoszewski’s jython-burp-api includes a an interactive Jython console injected into a Burp tab. While I don’t recommend developing new extensions against the unofficial extension-hosting-extensions that were around before the official Burp API (in 1.5.01), access to the Jython tab is a pretty killer feature that stands well on its own.

You can install the jython-burp-api just as with the demo extension in step 6 above. The extension assumes that the “Folder for loading modules” (from step 4 above) is set to its own Lib/ directory. If you get errors such as “ImportError: No module named gds“, then either temporarily change your module folder, or use the solution noted here to have the extension fix up its own path.

Once that’s working, it will add an interactive Jython shell tab into the Burp interface.

jython_interpreter1

This shell was originally intended to work with the classes and objects defined jython-burp-api, but it’s possible to pull back the curtain and get access to the exact same Burp API that you’re developing against in standalone extensions.

Within the pre-defined “Burp” object is a reference to the Callbacks object passed to every extension. From there, you can manually call any of the methods available to an extension. During development and testing of your own extensions, it can be very useful to manually try out code on a particular request (which you can access from the history via getProxyHistory() ). Once you figure out what works, then that code can go into your extension.

Objects from the official Burp Java API can be identified by their lack of help() strings and the obfuscated type names, but python-style exploratory programming still works as expected: the dir() function lists available fields and methods, which correspond to the Burp Java API.

Objects from the official Burp Java API can be identified by their lack of help() strings and the obfuscated type names, but python-style exploratory programming still works as expected: the dir() function lists available fields and methods, which correspond to the Burp Java API.

Testing Web Services With Burp and SoapUI

When assessing custom web services, what we often get from customers is simply a spec document, maybe with a few concrete examples peppered throughout; actual working sample code, or real proxy logs are a rare luxury. In these cases, it becomes useful to have an interface that will be able to help craft and replay messages, and easily support variations of the same message (“Message A (as documented in spec)”, “Message A (actually working)”, “testing injections into field x”, “testing parameter overload of field y”, etc). While Burp is excellent at replaying and tampering with existing requests, the Burp interface doesn’t do a great job of helping to craft entirely new messages, or helping keep dozens of different variations organized and documented.

For this task, I turn to a tool familiar to many developers, but rather less known among pentesters: SoapUI. SoapUI calls itself the “swiss army knife of [web service] testing”, and it does a pretty good job of living up to that. By proxying it through Burp (File>Preferences>Proxy Settings) and using Burp extensions to programmatically deal with any additional logic required for the service, you can use the strengths of both and have an excellent environment for security testing against services . SoapUI Pro even includes a variety of web-service specific payloads for security testing.

The main SoapUI interface, populated for penetration testing against a web service. Several variations of a particular service-specific POST message are shown, each demonstrating and providing easy reproducability for a discovered vulnerability.

The main SoapUI interface, populated for penetration testing against a web service. Several variations of a particular service-specific POST message are shown, each demonstrating and providing easy reproducability for a discovered vulnerability.

If the service offers a WSDL or WADL, configuring SoapUI to interact with it is straightforward; simply start a new project, paste in the URL of the endpoint, and hit okay. If the service is a REST service, or some other mechanism over HTTP, you can skip all of the validation checks and simply start manually creating requests by ticking the “Add REST Service” box in the “New SoapUI Project” dialog.

Create, manage, and send arbitrary HTTP requests without a “proper” WSDL or service description by telling SoapUI it’s a REST service.

Create, manage, and send arbitrary HTTP requests without a “proper” WSDL or service description by telling SoapUI it’s a REST service.

In addition to helping you create and send requests, I find that the soapUI project file is an invaluable resource for future assessments on the same service; any other tester can pick up right where I left off (even months or years later) by loading my Burp state and my SoapUI project file.

Share Back!

This should be enough to get you up and running with custom Burp extensions to handle unusual service logic, and SoapUI to craft and manage large numbers of example messages and payloads. For Burp, there are a tons of tools out there, including official Burp examples, burpextensions.com, and findable on github. Make sure to share other useful extensions, tools, or tricks in the comments, or hit me up to discuss: @coffeetocode or @neohapsis.

Charlie Brown’s Nightmare Before Christmas

Wednesday, January 26th, 2011

I always enjoy reading the Christmas Challenges created by Ed Skoudis and Yori Kvitchko over at ethicalhacker.net. This year’s puzzle was “The Nightmare Before Charlie Brown’s Christmas” and offered a chance to play around with VoIP, which I don’t get to do much of normally.

The winners were just posted, and my entry got the nod for Best Creative Entry. This is particularly awesome for me since the original Counter Hack (by Skoudis) was one of the first security books I ever bought.

I highly recommend reading through the contest and the answers; as always, the technical walkthrough is hugely informative, and they cover a massive toychest of wicked VoIP hacking utilities. There’s also some pretty nice command line kung foo (hat tip) that makes me remember the power of the Unix philosophy of small tools.