Fiddler is a very popular tool for web debugging. It has been out for a few years now and it has millions of users. In this article I will cover some of the basic features of Fiddler and in a future article I will build on this foundation and talk about more advanced features.
What is Fiddler?
Fiddler is a Web Debugging Proxy which logs all HTTP(S) traffic between your computer and the Internet. Fiddler allows you to inspect traffic, set breakpoints, and “fiddle” with incoming or outgoing data. Fiddler includes a powerful event-based scripting subsystem, and can be extended using any .NET language.
Fiddler is freeware and can debug traffic from virtually any application that supports a proxy, including Internet Explorer, Google Chrome, Apple Safari, Mozilla Firefox, Opera, and thousands more. You can also debug traffic from popular devices like Windows Phone, iPod/iPad, and others.
You can get the application from the Fiddler’s website along with tones of documentation, tutorial videos and so on. On the Fiddler’s website there is also a Fiddler Extension repository that lists major Fiddler’s extensions.
How does it work?
Fiddler sits between your http client and http server listening on a port for http(s) traffic.
WinINet (The Windows Internet) enables applications to interact with FTP, and HTTP protocols to access Internet resources.
I am sure you are familiar with ‘LAN Settings‘ under the ‘Connections‘ tab of Internet (Explorer) Options:
The Proxy Settings under ‘LAN Settings‘ is your computer’s proxy settings which is kind of like the UI for WinINet.
When you start up Fiddler it saves your current proxy settings and changes it so it can work as a proxy for all your http traffic:
And of course it sets back the proxy to your original settings when you close the Fiddler or stop it. You can start and stop Fiddler capturing through File -> Capture Traffic, or using F12, or using start and stop QuickExec commands.
Fiddler is a Web Proxy; so it does not really care where the traffic is coming from and where it is going to. This means that you can use Fiddler on pretty much any application or device that can talk to a proxy:
Fiddler is not the only tool that works as a proxy or provides these capabilities. Following is the list of some of the more or less similar tools:
- WireShark: a free and open-source packet analyzer. It is used for network troubleshooting, analysis, software and communications protocol development, and education.
- Microsoft Network Monitor (AKA NetMon): a packet analyzer. It enables capturing, viewing, and analyzing network data and deciphering network protocols. It can be used to troubleshoot network problems and applications on the network.
- Browser Plugins (they are not proxy)
- HttpWatch for IE and FireFox: an integrated HTTP sniffer for IE and Firefox that provides new insights into how your website loads and performs.
- Chrome developer tools (Ctrl+shift+I)
- FireBug Lite for Chrome: provides the rich visual representation we are used to see in Firebug when it comes to HTML elements, DOM elements, and Box Model shading. It provides also some cool features like inspecting HTML elemements with your mouse, and live editing CSS properties.
- IE Dev Tools (F12): IE9 currently provides some of the Fiddler’s feature and is going to provide even more – I guess because Eric Lawrence, the author of Fiddler, is a PM on IE team.
WireShark and NetMon are proxies which are more or less similar to Fiddler in nature; however, while Fiddler intercepts the traffic on the Application Layer, WireShark and NetMon intercept the traffic on much lower level and they are not limited to HTTP. This makes them awesome choices when it comes to low level debugging of network issues or monitoring protocols other than http; however, they do not display the traffic in an as easily understandable form as Fiddler.
Browser plugins are, however, different from proxies in nature. Firstly they run in the browser and as such can only intercept browser traffic. All the browser plugins I know limit the traffic monitoring to the session/tab they are running on (which makes sense). Proxies however run as a separate process and intercept traffic regardless of where it is coming from.
The other difference worth pointing out is that browsers serve (valid/current) cached contents locally without hitting the network. For example if your website requires a CSS file that has been cached on the computer from a previous session and is still valid, when you browse to the website again, the browser is going to serve the css locally. This means that the browser plugin is going to see the content served from the cache while proxies will not see the request and the traffic because browser does not make any request.
Let’s get started with Fiddler
Fiddler is a Windows Forms application written in .Net 2. The UI (at V18.104.22.168) looks like:
We are going to go through the UI and see what each tab/menu/feature does. Well, I do this for the most used/useful features and this by no means is an exhaustive reference. For more information please refer to Fiddler web site where Eric provides a lot of information on every single feature.
Web Sessions pane is where the traffic is displayed as it is captured:
Directly copied from Fiddler website: Certain key information is available in this list, including:
- # – An ID# of the request generated by Fiddler for your convenience
- Result – The Result code from the HTTP Response
- Protocol – The Protocol (HTTP/HTTPS/FTP) used by this session
- Host – The hostname of the server to which the request was sent
- URL – The path and file requested from the server
- Body – The number of bytes in the Response body
- Caching – Values from the Response’s Expires or Cache-Control headers
- Process – The local Windows Process from which the traffic originated
- Content-Type – The Content-Type header from the Response
- Custom – A text field you can set via scripting
- Comments – A text field you can set from scripting or the session’s context menu
The icons on the left hand side next to the ID give you more visual information about the session:
One little known awesome feature of Web Sessions is the ability to compare two sessions. To compare two sessions select them in the Web Sessions pane, right click and choose ‘Compare‘ item from the menu.
By default Fiddler uses WinDiff as diff tool; but you can change this in registry:
- Open Registry, go to HKEY_CURRENT_USER\Software\Microsoft\Fiddler2
- Add a new String Value called CompareTool and set the value to the path of your favorite Diff tool (e.g. “C:\Program Files\Beyond Compare 3\BCompare.exe”)
- Restart Fiddler
The first tab on the right pane is Statistics which shows you some info about the selected session. It is most useful when used on several sessions:
Is this not awesome?! So it is telling me that I have selected 26 requests, where total bytes sent and received are 166K and 197K (with more info about the header and body size), it provides a summary on received status codes and MIME types. And then there is the nice chart on the bottom.
Inspectors allow you to visualize requests and responses in meaningful ways. There are heaps of out of the box inspectors and you can write your own if you need to.
When you double click on a session fiddler takes you to the Inspectors tab for that session. It also detects what the most relevant request and response inspectors are and chooses that for you. For example if you double click on a JSON session, Fiddler takes you to the JSON tab.
There are many inspectors and I could write a 30 page article about it; but it is much easier to understand if you run Fiddler and start double clicking on sessions.
HTTPS Traffic Decryption
Fiddler can decipher HTTPS traffic. This scares some of the guys at security/I.T. department; but this is nothing to worry about. It can only do so for the traffic going through it and by your permission.
By default Fiddler does not decipher HTTPS traffic and you should explicitly ask it to do so. In order to activate HTTPS traffic decryption:
Go to Tools -> Fiddler Options -> HTTPS tab
Tick Capture HTTPS CONNECTs and Decrypt HTTPS traffic checkboxes.
After ticking the second checkbox you get an alarming pop up message:
Clicking yes on that dialog brings up a second warning, this time by Windows.
Clicking yes on this box installs Fiddler’s certificate in your Personal store:
Fiddler uses a hacking technique called Man-In-The-Middle Attack to decrypt HTTPS traffic:
- The client thinks it is talking to the server while it is talking to the man-in-the-middle (MITM) and it uses the MITM’s certificate for SSL.
- MITM then receives client’s request and potentially modifies it and sends it to the server using the server’s certificate.
- Server receives the request thinking it came from the client, does the processing and returns the response to the MITM thinking it is the client.
- MITM receives the response and optionally modifies it and returns it to the client.
So while the client and server are unaware of MITM, the whole traffic has been monitored and perhaps modified. Fiddler does pretty much the same thing except that you know it is sitting in the middle.