PHChapter 1: The Structure of HTTP Requests
35
Chapter 2: Examining HTTP Traffic
73
Chapter 3: Simple Requests
93
Chapter 4: Beyond Simple Requests
127
Chapter 5: Secure HTTP Requests
153
Chapter 6: Extracting Data
171
Chapter 7: Responding to Forms
251
Chapter 8: Handling Sessions and Cookies
285
Chapter 9: Using JavaScript
307
Chapter 10: Working with AJAX Sites
339
Chapter 11: Handling Web Services
365
Chapter 12: Working with RSS Feeds
395
Chapter 13: Using a Spider
419
Chapter 14: Inside the Heaton Research Spider
459
Chapter 15: Using a SQL Workload
519
Chapter 16: Well Behaved Bots
563
Appendix A: Downloading Examples
581
Appendix B: Setting Up Examples
585
Appendix C: Useful Classes, Methods and Functions
591
Appendix D: Setting Up your Database
595
Appendix E: HTTP Response Codes
601
XIV HTTP Programming Recipes for C# Bots
XV
Contents
Introduction ...XXXI
Chapter 1: The Structure of HTTP Requests
35
The HTTP Recipes Examples Site
35
The Structure of Surfing
38
Examining HTTP Requests
38
HTTP Requests and Response
42
HTTP Headers
47
Recipes ...50
Summary ...71
Chapter 2: Examining HTTP Traffic
73
Using a Network Analyzer
73
Understanding WireShark
74
Recipes ...85
Summary ...90
Chapter 3: Simple Requests
93
Constructing URLs
94
What is a URL?
95
Encoding Special Characters into a URL
97
Reading from URLs
98
Recipes ...101
Summary ...124
Chapter 4: Beyond Simple Requests
127
Using HttpWebRequest
127
Using HttpWebResponse
130
Other Useful Options
132
Recipes ...133
Summary ...150
Chapter 5: Secure HTTP Requests
153
Using HTTPS in C#
153
Understanding HTTP Authentication
158
Recipes ...161
Summary ...167
Chapter 6: Extracting Data
171
Peekable Stream
172
Parsing HTML
179
Encapsulating HTML Tags
200
Contents
XVI HTTP Programming Recipes for C# Bots
Recipes ...205
Summary ...248
Chapter 7: Responding to Forms
251
Understanding HTML Forms
251
POST or GET
256
Multipart POST
258
Processing Forms
261
Recipes ...268
Summary ...282
Chapter 8: Handling Sessions and Cookies
285
URL Variables for State
285
Cookies for State
287
Recipes ...289
Summary ...305
Chapter 9: Using JavaScript
307
Understanding JavaScript
307
Common JavaScript Techniques
308
Interpreting JavaScript
317
Recipes ...318
Summary ...335
Chapter 10: Working with AJAX Sites
339
Understanding AJAX
340
Recipes ...346
Summary ...362
Chapter 11: Handling Web Services
365
Notable Public Web Services
365
Using the Google API
366
Hybrid Bots
367
Understanding SOAP
367
Recipes ...375
Summary ...392
Chapter 12: Working with RSS Feeds
395
Using RSS with a Web Browser
395
RSS Format
397
Parsing RSS Files
401
Recipes ...411
Summary ...415
Chapter 13: Using a Spider
419
XVII
Using the Heaton Research Spider
420
Recipes ...428
Summary ...456
Chapter 14: Inside the Heaton Research Spider
459
The Spider Class
461
Other Important Classes in the Heaton Research Spider
475
Workload Management
498
Implementing a Memory Based WorkloadManager
504
Summary
516
Chapter 15: Using a SQL Workload
519
Repeatable Statements
520
Implementing a SQL Based Workload Manager
531
Summary
560
Chapter 16: Well Behaved Bots
563
Using a CAPTCHA
564
User Agent Filtering
566
Robots Exclusion Standard
567
Using Filters with the Heaton Research Spider
569
Implementing a robots.txt Filter
570
Summary
579
Appendix A: Downloading Examples
581
Appendix B: Setting Up Examples
585
Running from a Command Prompt
586
Running from Visual Studio
587
Compiling without Visual Studio
588
Appendix C: Useful Classes, Methods and Functions
591
Reusable Functions and Methods
591
Reusable Classes
591
All Recipes
592
Appendix D: Setting Up your Database
595
DDL for MySQL
595
DDL for Microsoft Access
596
DDL for Oracle
597
OLEDB for .NET
599
Appendix E: HTTP Response Codes
601
1xx Informational
601
2xx Success
601
3xx Redirection
601
XVIII HTTP Programming Recipes for C# Bots
4xx Client Error
602
5xx Server Error
Download link:
You do not have permission to view the full content of this post. Log in or register now.
35
Chapter 2: Examining HTTP Traffic
73
Chapter 3: Simple Requests
93
Chapter 4: Beyond Simple Requests
127
Chapter 5: Secure HTTP Requests
153
Chapter 6: Extracting Data
171
Chapter 7: Responding to Forms
251
Chapter 8: Handling Sessions and Cookies
285
Chapter 9: Using JavaScript
307
Chapter 10: Working with AJAX Sites
339
Chapter 11: Handling Web Services
365
Chapter 12: Working with RSS Feeds
395
Chapter 13: Using a Spider
419
Chapter 14: Inside the Heaton Research Spider
459
Chapter 15: Using a SQL Workload
519
Chapter 16: Well Behaved Bots
563
Appendix A: Downloading Examples
581
Appendix B: Setting Up Examples
585
Appendix C: Useful Classes, Methods and Functions
591
Appendix D: Setting Up your Database
595
Appendix E: HTTP Response Codes
601
XIV HTTP Programming Recipes for C# Bots
XV
Contents
Introduction ...XXXI
Chapter 1: The Structure of HTTP Requests
35
The HTTP Recipes Examples Site
35
The Structure of Surfing
38
Examining HTTP Requests
38
HTTP Requests and Response
42
HTTP Headers
47
Recipes ...50
Summary ...71
Chapter 2: Examining HTTP Traffic
73
Using a Network Analyzer
73
Understanding WireShark
74
Recipes ...85
Summary ...90
Chapter 3: Simple Requests
93
Constructing URLs
94
What is a URL?
95
Encoding Special Characters into a URL
97
Reading from URLs
98
Recipes ...101
Summary ...124
Chapter 4: Beyond Simple Requests
127
Using HttpWebRequest
127
Using HttpWebResponse
130
Other Useful Options
132
Recipes ...133
Summary ...150
Chapter 5: Secure HTTP Requests
153
Using HTTPS in C#
153
Understanding HTTP Authentication
158
Recipes ...161
Summary ...167
Chapter 6: Extracting Data
171
Peekable Stream
172
Parsing HTML
179
Encapsulating HTML Tags
200
Contents
XVI HTTP Programming Recipes for C# Bots
Recipes ...205
Summary ...248
Chapter 7: Responding to Forms
251
Understanding HTML Forms
251
POST or GET
256
Multipart POST
258
Processing Forms
261
Recipes ...268
Summary ...282
Chapter 8: Handling Sessions and Cookies
285
URL Variables for State
285
Cookies for State
287
Recipes ...289
Summary ...305
Chapter 9: Using JavaScript
307
Understanding JavaScript
307
Common JavaScript Techniques
308
Interpreting JavaScript
317
Recipes ...318
Summary ...335
Chapter 10: Working with AJAX Sites
339
Understanding AJAX
340
Recipes ...346
Summary ...362
Chapter 11: Handling Web Services
365
Notable Public Web Services
365
Using the Google API
366
Hybrid Bots
367
Understanding SOAP
367
Recipes ...375
Summary ...392
Chapter 12: Working with RSS Feeds
395
Using RSS with a Web Browser
395
RSS Format
397
Parsing RSS Files
401
Recipes ...411
Summary ...415
Chapter 13: Using a Spider
419
XVII
Using the Heaton Research Spider
420
Recipes ...428
Summary ...456
Chapter 14: Inside the Heaton Research Spider
459
The Spider Class
461
Other Important Classes in the Heaton Research Spider
475
Workload Management
498
Implementing a Memory Based WorkloadManager
504
Summary
516
Chapter 15: Using a SQL Workload
519
Repeatable Statements
520
Implementing a SQL Based Workload Manager
531
Summary
560
Chapter 16: Well Behaved Bots
563
Using a CAPTCHA
564
User Agent Filtering
566
Robots Exclusion Standard
567
Using Filters with the Heaton Research Spider
569
Implementing a robots.txt Filter
570
Summary
579
Appendix A: Downloading Examples
581
Appendix B: Setting Up Examples
585
Running from a Command Prompt
586
Running from Visual Studio
587
Compiling without Visual Studio
588
Appendix C: Useful Classes, Methods and Functions
591
Reusable Functions and Methods
591
Reusable Classes
591
All Recipes
592
Appendix D: Setting Up your Database
595
DDL for MySQL
595
DDL for Microsoft Access
596
DDL for Oracle
597
OLEDB for .NET
599
Appendix E: HTTP Response Codes
601
1xx Informational
601
2xx Success
601
3xx Redirection
601
XVIII HTTP Programming Recipes for C# Bots
4xx Client Error
602
5xx Server Error
Download link:
You do not have permission to view the full content of this post. Log in or register now.
)
