Python2/PyMuPDF: mupdf-source/thirdparty/curl/docs/TheArtOfHttpScripting annotate

annotate mupdf-source/thirdparty/curl/docs/TheArtOfHttpScripting @ 2:b50eed0cc0ef upstream

ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. The directory name has changed: no version number in the expanded directory now.

author	Franz Glasner <fzglas.hg@dom66.de>
date	Mon, 15 Sep 2025 11:43:07 +0200
parents
children

rev	line source
2 b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	1 _ _ ____ _
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	2 ___\| \| \| \| _ \\| \|
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	3 / __\| \| \| \| \|_) \| \|
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	4 \| (__\| \|_\| \| _ <\| \|___
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	5 \___\|\___/\|_\| \_\_____\|
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	6
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	7
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	8 The Art Of Scripting HTTP Requests Using Curl
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	9
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	10 1. HTTP Scripting
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	11 1.1 Background
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	12 1.2 The HTTP Protocol
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	13 1.3 See the Protocol
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	14 1.4 See the Timing
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	15 1.5 See the Response
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	16 2. URL
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	17 2.1 Spec
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	18 2.2 Host
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	19 2.3 Port number
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	20 2.4 User name and password
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	21 2.5 Path part
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	22 3. Fetch a page
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	23 3.1 GET
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	24 3.2 HEAD
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	25 3.3 Multiple URLs in a single command line
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	26 3.4 Multiple HTTP methods in a single command line
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	27 4. HTML forms
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	28 4.1 Forms explained
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	29 4.2 GET
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	30 4.3 POST
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	31 4.4 File Upload POST
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	32 4.5 Hidden Fields
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	33 4.6 Figure Out What A POST Looks Like
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	34 5. HTTP upload
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	35 5.1 PUT
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	36 6. HTTP Authentication
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	37 6.1 Basic Authentication
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	38 6.2 Other Authentication
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	39 6.3 Proxy Authentication
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	40 6.4 Hiding credentials
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	41 7. More HTTP Headers
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	42 7.1 Referer
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	43 7.2 User Agent
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	44 8. Redirects
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	45 8.1 Location header
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	46 8.2 Other redirects
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	47 9. Cookies
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	48 9.1 Cookie Basics
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	49 9.2 Cookie options
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	50 10. HTTPS
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	51 10.1 HTTPS is HTTP secure
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	52 10.2 Certificates
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	53 11. Custom Request Elements
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	54 11.1 Modify method and headers
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	55 11.2 More on changed methods
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	56 12. Web Login
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	57 12.1 Some login tricks
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	58 13. Debug
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	59 13.1 Some debug tricks
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	60 14. References
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	61 14.1 Standards
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	62 14.2 Sites
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	63
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	64 ==============================================================================
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	65
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	66 1. HTTP Scripting
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	67
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	68 1.1 Background
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	69
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	70 This document assumes that you're familiar with HTML and general networking.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	71
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	72 The increasing amount of applications moving to the web has made "HTTP
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	73 Scripting" more frequently requested and wanted. To be able to automatically
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	74 extract information from the web, to fake users, to post or upload data to
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	75 web servers are all important tasks today.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	76
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	77 Curl is a command line tool for doing all sorts of URL manipulations and
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	78 transfers, but this particular document will focus on how to use it when
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	79 doing HTTP requests for fun and profit. I'll assume that you know how to
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	80 invoke 'curl --help' or 'curl --manual' to get basic information about it.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	81
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	82 Curl is not written to do everything for you. It makes the requests, it gets
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	83 the data, it sends data and it retrieves the information. You probably need
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	84 to glue everything together using some kind of script language or repeated
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	85 manual invokes.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	86
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	87 1.2 The HTTP Protocol
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	88
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	89 HTTP is the protocol used to fetch data from web servers. It is a very simple
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	90 protocol that is built upon TCP/IP. The protocol also allows information to
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	91 get sent to the server from the client using a few different methods, as will
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	92 be shown here.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	93
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	94 HTTP is plain ASCII text lines being sent by the client to a server to
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	95 request a particular action, and then the server replies a few text lines
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	96 before the actual requested content is sent to the client.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	97
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	98 The client, curl, sends a HTTP request. The request contains a method (like
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	99 GET, POST, HEAD etc), a number of request headers and sometimes a request
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	100 body. The HTTP server responds with a status line (indicating if things went
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	101 well), response headers and most often also a response body. The "body" part
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	102 is the plain data you requested, like the actual HTML or the image etc.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	103
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	104 1.3 See the Protocol
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	105
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	106 Using curl's option --verbose (-v as a short option) will display what kind
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	107 of commands curl sends to the server, as well as a few other informational
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	108 texts.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	109
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	110 --verbose is the single most useful option when it comes to debug or even
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	111 understand the curl<->server interaction.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	112
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	113 Sometimes even --verbose is not enough. Then --trace and --trace-ascii offer
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	114 even more details as they show EVERYTHING curl sends and receives. Use it
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	115 like this:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	116
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	117 curl --trace-ascii debugdump.txt http://www.example.com/
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	118
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	119 1.4 See the Timing
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	120
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	121 Many times you may wonder what exactly is taking all the time, or you just
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	122 want to know the amount of milliseconds between two points in a
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	123 transfer. For those, and other similar situations, the --trace-time option
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	124 is what you need. It'll prepend the time to each trace output line:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	125
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	126 curl --trace-ascii d.txt --trace-time http://example.com/
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	127
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	128 1.5 See the Response
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	129
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	130 By default curl sends the response to stdout. You need to redirect it
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	131 somewhere to avoid that, most often that is done with -o or -O.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	132
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	133 2. URL
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	134
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	135 2.1 Spec
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	136
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	137 The Uniform Resource Locator format is how you specify the address of a
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	138 particular resource on the Internet. You know these, you've seen URLs like
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	139 https://curl.haxx.se or https://yourbank.com a million times. RFC 3986 is the
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	140 canonical spec. And yeah, the formal name is not URL, it is URI.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	141
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	142 2.2 Host
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	143
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	144 The host name is usually resolved using DNS or your /etc/hosts file to an IP
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	145 address and that's what curl will communicate with. Alternatively you specify
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	146 the IP address directly in the URL instead of a name.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	147
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	148 For development and other trying out situations, you can point to a different
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	149 IP address for a host name than what would otherwise be used, by using curl's
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	150 --resolve option:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	151
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	152 curl --resolve www.example.org:80:127.0.0.1 http://www.example.org/
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	153
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	154 2.3 Port number
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	155
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	156 Each protocol curl supports operates on a default port number, be it over TCP
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	157 or in some cases UDP. Normally you don't have to take that into
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	158 consideration, but at times you run test servers on other ports or
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	159 similar. Then you can specify the port number in the URL with a colon and a
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	160 number immediately following the host name. Like when doing HTTP to port
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	161 1234:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	162
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	163 curl http://www.example.org:1234/
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	164
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	165 The port number you specify in the URL is the number that the server uses to
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	166 offer its services. Sometimes you may use a local proxy, and then you may
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	167 need to specify that proxy's port number separately for what curl needs to
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	168 connect to locally. Like when using a HTTP proxy on port 4321:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	169
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	170 curl --proxy http://proxy.example.org:4321 http://remote.example.org/
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	171
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	172 2.4 User name and password
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	173
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	174 Some services are setup to require HTTP authentication and then you need to
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	175 provide name and password which is then transferred to the remote site in
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	176 various ways depending on the exact authentication protocol used.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	177
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	178 You can opt to either insert the user and password in the URL or you can
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	179 provide them separately:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	180
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	181 curl http://user:password@example.org/
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	182
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	183 or
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	184
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	185 curl -u user:password http://example.org/
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	186
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	187 You need to pay attention that this kind of HTTP authentication is not what
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	188 is usually done and requested by user-oriented web sites these days. They
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	189 tend to use forms and cookies instead.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	190
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	191 2.5 Path part
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	192
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	193 The path part is just sent off to the server to request that it sends back
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	194 the associated response. The path is what is to the right side of the slash
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	195 that follows the host name and possibly port number.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	196
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	197 3. Fetch a page
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	198
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	199 3.1 GET
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	200
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	201 The simplest and most common request/operation made using HTTP is to GET a
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	202 URL. The URL could itself refer to a web page, an image or a file. The client
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	203 issues a GET request to the server and receives the document it asked for.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	204 If you issue the command line
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	205
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	206 curl https://curl.haxx.se
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	207
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	208 you get a web page returned in your terminal window. The entire HTML document
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	209 that that URL holds.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	210
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	211 All HTTP replies contain a set of response headers that are normally hidden,
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	212 use curl's --include (-i) option to display them as well as the rest of the
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	213 document.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	214
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	215 3.2 HEAD
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	216
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	217 You can ask the remote server for ONLY the headers by using the --head (-I)
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	218 option which will make curl issue a HEAD request. In some special cases
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	219 servers deny the HEAD method while others still work, which is a particular
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	220 kind of annoyance.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	221
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	222 The HEAD method is defined and made so that the server returns the headers
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	223 exactly the way it would do for a GET, but without a body. It means that you
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	224 may see a Content-Length: in the response headers, but there must not be an
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	225 actual body in the HEAD response.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	226
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	227 3.3 Multiple URLs in a single command line
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	228
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	229 A single curl command line may involve one or many URLs. The most common case
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	230 is probably to just use one, but you can specify any amount of URLs. Yes
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	231 any. No limits. You'll then get requests repeated over and over for all the
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	232 given URLs.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	233
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	234 Example, send two GETs:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	235
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	236 curl http://url1.example.com http://url2.example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	237
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	238 If you use --data to POST to the URL, using multiple URLs means that you send
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	239 that same POST to all the given URLs.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	240
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	241 Example, send two POSTs:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	242
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	243 curl --data name=curl http://url1.example.com http://url2.example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	244
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	245
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	246 3.4 Multiple HTTP methods in a single command line
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	247
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	248 Sometimes you need to operate on several URLs in a single command line and do
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	249 different HTTP methods on each. For this, you'll enjoy the --next option. It
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	250 is basically a separator that separates a bunch of options from the next. All
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	251 the URLs before --next will get the same method and will get all the POST
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	252 data merged into one.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	253
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	254 When curl reaches the --next on the command line, it'll sort of reset the
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	255 method and the POST data and allow a new set.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	256
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	257 Perhaps this is best shown with a few examples. To send first a HEAD and then
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	258 a GET:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	259
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	260 curl -I http://example.com --next http://example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	261
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	262 To first send a POST and then a GET:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	263
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	264 curl -d score=10 http://example.com/post.cgi --next http://example.com/results.html
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	265
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	266
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	267 4. HTML forms
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	268
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	269 4.1 Forms explained
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	270
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	271 Forms are the general way a web site can present a HTML page with fields for
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	272 the user to enter data in, and then press some kind of 'OK' or 'Submit'
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	273 button to get that data sent to the server. The server then typically uses
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	274 the posted data to decide how to act. Like using the entered words to search
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	275 in a database, or to add the info in a bug tracking system, display the entered
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	276 address on a map or using the info as a login-prompt verifying that the user
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	277 is allowed to see what it is about to see.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	278
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	279 Of course there has to be some kind of program on the server end to receive
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	280 the data you send. You cannot just invent something out of the air.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	281
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	282 4.2 GET
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	283
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	284 A GET-form uses the method GET, as specified in HTML like:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	285
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	286 <form method="GET" action="junk.cgi">
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	287 <input type=text name="birthyear">
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	288 <input type=submit name=press value="OK">
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	289 </form>
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	290
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	291 In your favorite browser, this form will appear with a text box to fill in
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	292 and a press-button labeled "OK". If you fill in '1905' and press the OK
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	293 button, your browser will then create a new URL to get for you. The URL will
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	294 get "junk.cgi?birthyear=1905&press=OK" appended to the path part of the
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	295 previous URL.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	296
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	297 If the original form was seen on the page "www.hotmail.com/when/birth.html",
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	298 the second page you'll get will become
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	299 "www.hotmail.com/when/junk.cgi?birthyear=1905&press=OK".
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	300
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	301 Most search engines work this way.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	302
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	303 To make curl do the GET form post for you, just enter the expected created
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	304 URL:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	305
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	306 curl "http://www.hotmail.com/when/junk.cgi?birthyear=1905&press=OK"
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	307
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	308 4.3 POST
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	309
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	310 The GET method makes all input field names get displayed in the URL field of
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	311 your browser. That's generally a good thing when you want to be able to
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	312 bookmark that page with your given data, but it is an obvious disadvantage
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	313 if you entered secret information in one of the fields or if there are a
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	314 large amount of fields creating a very long and unreadable URL.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	315
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	316 The HTTP protocol then offers the POST method. This way the client sends the
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	317 data separated from the URL and thus you won't see any of it in the URL
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	318 address field.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	319
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	320 The form would look very similar to the previous one:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	321
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	322 <form method="POST" action="junk.cgi">
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	323 <input type=text name="birthyear">
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	324 <input type=submit name=press value=" OK ">
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	325 </form>
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	326
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	327 And to use curl to post this form with the same data filled in as before, we
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	328 could do it like:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	329
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	330 curl --data "birthyear=1905&press=%20OK%20" \
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	331 http://www.example.com/when.cgi
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	332
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	333 This kind of POST will use the Content-Type
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	334 application/x-www-form-urlencoded and is the most widely used POST kind.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	335
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	336 The data you send to the server MUST already be properly encoded, curl will
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	337 not do that for you. For example, if you want the data to contain a space,
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	338 you need to replace that space with %20 etc. Failing to comply with this
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	339 will most likely cause your data to be received wrongly and messed up.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	340
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	341 Recent curl versions can in fact url-encode POST data for you, like this:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	342
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	343 curl --data-urlencode "name=I am Daniel" http://www.example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	344
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	345 If you repeat --data several times on the command line, curl will
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	346 concatenate all the given data pieces - and put a '&' symbol between each
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	347 data segment.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	348
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	349 4.4 File Upload POST
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	350
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	351 Back in late 1995 they defined an additional way to post data over HTTP. It
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	352 is documented in the RFC 1867, why this method sometimes is referred to as
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	353 RFC1867-posting.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	354
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	355 This method is mainly designed to better support file uploads. A form that
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	356 allows a user to upload a file could be written like this in HTML:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	357
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	358 <form method="POST" enctype='multipart/form-data' action="upload.cgi">
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	359 <input type=file name=upload>
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	360 <input type=submit name=press value="OK">
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	361 </form>
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	362
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	363 This clearly shows that the Content-Type about to be sent is
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	364 multipart/form-data.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	365
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	366 To post to a form like this with curl, you enter a command line like:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	367
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	368 curl --form upload=@localfilename --form press=OK [URL]
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	369
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	370 4.5 Hidden Fields
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	371
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	372 A very common way for HTML based applications to pass state information
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	373 between pages is to add hidden fields to the forms. Hidden fields are
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	374 already filled in, they aren't displayed to the user and they get passed
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	375 along just as all the other fields.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	376
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	377 A similar example form with one visible field, one hidden field and one
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	378 submit button could look like:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	379
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	380 <form method="POST" action="foobar.cgi">
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	381 <input type=text name="birthyear">
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	382 <input type=hidden name="person" value="daniel">
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	383 <input type=submit name="press" value="OK">
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	384 </form>
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	385
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	386 To POST this with curl, you won't have to think about if the fields are
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	387 hidden or not. To curl they're all the same:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	388
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	389 curl --data "birthyear=1905&press=OK&person=daniel" [URL]
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	390
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	391 4.6 Figure Out What A POST Looks Like
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	392
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	393 When you're about fill in a form and send to a server by using curl instead
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	394 of a browser, you're of course very interested in sending a POST exactly the
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	395 way your browser does.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	396
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	397 An easy way to get to see this, is to save the HTML page with the form on
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	398 your local disk, modify the 'method' to a GET, and press the submit button
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	399 (you could also change the action URL if you want to).
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	400
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	401 You will then clearly see the data get appended to the URL, separated with a
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	402 '?'-letter as GET forms are supposed to.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	403
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	404 5. HTTP upload
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	405
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	406 5.1 PUT
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	407
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	408 Perhaps the best way to upload data to a HTTP server is to use PUT. Then
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	409 again, this of course requires that someone put a program or script on the
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	410 server end that knows how to receive a HTTP PUT stream.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	411
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	412 Put a file to a HTTP server with curl:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	413
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	414 curl --upload-file uploadfile http://www.example.com/receive.cgi
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	415
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	416 6. HTTP Authentication
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	417
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	418 6.1 Basic Authentication
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	419
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	420 HTTP Authentication is the ability to tell the server your username and
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	421 password so that it can verify that you're allowed to do the request you're
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	422 doing. The Basic authentication used in HTTP (which is the type curl uses by
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	423 default) is plain text based, which means it sends username and password
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	424 only slightly obfuscated, but still fully readable by anyone that sniffs on
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	425 the network between you and the remote server.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	426
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	427 To tell curl to use a user and password for authentication:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	428
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	429 curl --user name:password http://www.example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	430
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	431 6.2 Other Authentication
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	432
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	433 The site might require a different authentication method (check the headers
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	434 returned by the server), and then --ntlm, --digest, --negotiate or even
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	435 --anyauth might be options that suit you.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	436
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	437 6.3 Proxy Authentication
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	438
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	439 Sometimes your HTTP access is only available through the use of a HTTP
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	440 proxy. This seems to be especially common at various companies. A HTTP proxy
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	441 may require its own user and password to allow the client to get through to
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	442 the Internet. To specify those with curl, run something like:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	443
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	444 curl --proxy-user proxyuser:proxypassword curl.haxx.se
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	445
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	446 If your proxy requires the authentication to be done using the NTLM method,
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	447 use --proxy-ntlm, if it requires Digest use --proxy-digest.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	448
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	449 If you use any one of these user+password options but leave out the password
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	450 part, curl will prompt for the password interactively.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	451
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	452 6.4 Hiding credentials
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	453
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	454 Do note that when a program is run, its parameters might be possible to see
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	455 when listing the running processes of the system. Thus, other users may be
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	456 able to watch your passwords if you pass them as plain command line
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	457 options. There are ways to circumvent this.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	458
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	459 It is worth noting that while this is how HTTP Authentication works, very
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	460 many web sites will not use this concept when they provide logins etc. See
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	461 the Web Login chapter further below for more details on that.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	462
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	463 7. More HTTP Headers
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	464
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	465 7.1 Referer
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	466
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	467 A HTTP request may include a 'referer' field (yes it is misspelled), which
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	468 can be used to tell from which URL the client got to this particular
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	469 resource. Some programs/scripts check the referer field of requests to verify
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	470 that this wasn't arriving from an external site or an unknown page. While
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	471 this is a stupid way to check something so easily forged, many scripts still
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	472 do it. Using curl, you can put anything you want in the referer-field and
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	473 thus more easily be able to fool the server into serving your request.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	474
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	475 Use curl to set the referer field with:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	476
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	477 curl --referer http://www.example.come http://www.example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	478
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	479 7.2 User Agent
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	480
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	481 Very similar to the referer field, all HTTP requests may set the User-Agent
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	482 field. It names what user agent (client) that is being used. Many
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	483 applications use this information to decide how to display pages. Silly web
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	484 programmers try to make different pages for users of different browsers to
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	485 make them look the best possible for their particular browsers. They usually
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	486 also do different kinds of javascript, vbscript etc.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	487
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	488 At times, you will see that getting a page with curl will not return the same
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	489 page that you see when getting the page with your browser. Then you know it
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	490 is time to set the User Agent field to fool the server into thinking you're
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	491 one of those browsers.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	492
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	493 To make curl look like Internet Explorer 5 on a Windows 2000 box:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	494
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	495 curl --user-agent "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)" [URL]
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	496
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	497 Or why not look like you're using Netscape 4.73 on an old Linux box:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	498
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	499 curl --user-agent "Mozilla/4.73 [en] (X11; U; Linux 2.2.15 i686)" [URL]
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	500
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	501 8. Redirects
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	502
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	503 8.1 Location header
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	504
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	505 When a resource is requested from a server, the reply from the server may
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	506 include a hint about where the browser should go next to find this page, or a
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	507 new page keeping newly generated output. The header that tells the browser
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	508 to redirect is Location:.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	509
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	510 Curl does not follow Location: headers by default, but will simply display
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	511 such pages in the same manner it displays all HTTP replies. It does however
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	512 feature an option that will make it attempt to follow the Location: pointers.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	513
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	514 To tell curl to follow a Location:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	515
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	516 curl --location http://www.example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	517
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	518 If you use curl to POST to a site that immediately redirects you to another
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	519 page, you can safely use --location (-L) and --data/--form together. Curl will
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	520 only use POST in the first request, and then revert to GET in the following
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	521 operations.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	522
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	523 8.2 Other redirects
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	524
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	525 Browser typically support at least two other ways of redirects that curl
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	526 doesn't: first the html may contain a meta refresh tag that asks the browser
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	527 to load a specific URL after a set number of seconds, or it may use
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	528 javascript to do it.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	529
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	530 9. Cookies
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	531
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	532 9.1 Cookie Basics
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	533
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	534 The way the web browsers do "client side state control" is by using
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	535 cookies. Cookies are just names with associated contents. The cookies are
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	536 sent to the client by the server. The server tells the client for what path
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	537 and host name it wants the cookie sent back, and it also sends an expiration
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	538 date and a few more properties.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	539
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	540 When a client communicates with a server with a name and path as previously
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	541 specified in a received cookie, the client sends back the cookies and their
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	542 contents to the server, unless of course they are expired.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	543
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	544 Many applications and servers use this method to connect a series of requests
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	545 into a single logical session. To be able to use curl in such occasions, we
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	546 must be able to record and send back cookies the way the web application
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	547 expects them. The same way browsers deal with them.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	548
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	549 9.2 Cookie options
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	550
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	551 The simplest way to send a few cookies to the server when getting a page with
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	552 curl is to add them on the command line like:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	553
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	554 curl --cookie "name=Daniel" http://www.example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	555
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	556 Cookies are sent as common HTTP headers. This is practical as it allows curl
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	557 to record cookies simply by recording headers. Record cookies with curl by
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	558 using the --dump-header (-D) option like:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	559
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	560 curl --dump-header headers_and_cookies http://www.example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	561
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	562 (Take note that the --cookie-jar option described below is a better way to
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	563 store cookies.)
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	564
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	565 Curl has a full blown cookie parsing engine built-in that comes in use if you
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	566 want to reconnect to a server and use cookies that were stored from a
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	567 previous connection (or hand-crafted manually to fool the server into
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	568 believing you had a previous connection). To use previously stored cookies,
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	569 you run curl like:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	570
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	571 curl --cookie stored_cookies_in_file http://www.example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	572
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	573 Curl's "cookie engine" gets enabled when you use the --cookie option. If you
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	574 only want curl to understand received cookies, use --cookie with a file that
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	575 doesn't exist. Example, if you want to let curl understand cookies from a
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	576 page and follow a location (and thus possibly send back cookies it received),
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	577 you can invoke it like:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	578
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	579 curl --cookie nada --location http://www.example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	580
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	581 Curl has the ability to read and write cookie files that use the same file
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	582 format that Netscape and Mozilla once used. It is a convenient way to share
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	583 cookies between scripts or invokes. The --cookie (-b) switch automatically
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	584 detects if a given file is such a cookie file and parses it, and by using the
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	585 --cookie-jar (-c) option you'll make curl write a new cookie file at the end
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	586 of an operation:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	587
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	588 curl --cookie cookies.txt --cookie-jar newcookies.txt \
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	589 http://www.example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	590
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	591 10. HTTPS
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	592
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	593 10.1 HTTPS is HTTP secure
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	594
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	595 There are a few ways to do secure HTTP transfers. By far the most common
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	596 protocol for doing this is what is generally known as HTTPS, HTTP over
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	597 SSL. SSL encrypts all the data that is sent and received over the network and
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	598 thus makes it harder for attackers to spy on sensitive information.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	599
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	600 SSL (or TLS as the latest version of the standard is called) offers a
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	601 truckload of advanced features to allow all those encryptions and key
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	602 infrastructure mechanisms encrypted HTTP requires.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	603
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	604 Curl supports encrypted fetches when built to use a TLS library and it can be
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	605 built to use one out of a fairly large set of libraries - "curl -V" will show
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	606 which one your curl was built to use (if any!). To get a page from a HTTPS
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	607 server, simply run curl like:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	608
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	609 curl https://secure.example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	610
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	611 10.2 Certificates
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	612
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	613 In the HTTPS world, you use certificates to validate that you are the one
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	614 you claim to be, as an addition to normal passwords. Curl supports client-
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	615 side certificates. All certificates are locked with a pass phrase, which you
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	616 need to enter before the certificate can be used by curl. The pass phrase
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	617 can be specified on the command line or if not, entered interactively when
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	618 curl queries for it. Use a certificate with curl on a HTTPS server like:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	619
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	620 curl --cert mycert.pem https://secure.example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	621
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	622 curl also tries to verify that the server is who it claims to be, by
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	623 verifying the server's certificate against a locally stored CA cert
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	624 bundle. Failing the verification will cause curl to deny the connection. You
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	625 must then use --insecure (-k) in case you want to tell curl to ignore that
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	626 the server can't be verified.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	627
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	628 More about server certificate verification and ca cert bundles can be read
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	629 in the SSLCERTS document, available online here:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	630
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	631 https://curl.haxx.se/docs/sslcerts.html
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	632
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	633 At times you may end up with your own CA cert store and then you can tell
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	634 curl to use that to verify the server's certificate:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	635
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	636 curl --cacert ca-bundle.pem https://example.com/
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	637
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	638
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	639 11. Custom Request Elements
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	640
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	641 11.1 Modify method and headers
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	642
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	643 Doing fancy stuff, you may need to add or change elements of a single curl
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	644 request.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	645
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	646 For example, you can change the POST request to a PROPFIND and send the data
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	647 as "Content-Type: text/xml" (instead of the default Content-Type) like this:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	648
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	649 curl --data "<xml>" --header "Content-Type: text/xml" \
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	650 --request PROPFIND url.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	651
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	652 You can delete a default header by providing one without content. Like you
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	653 can ruin the request by chopping off the Host: header:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	654
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	655 curl --header "Host:" http://www.example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	656
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	657 You can add headers the same way. Your server may want a "Destination:"
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	658 header, and you can add it:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	659
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	660 curl --header "Destination: http://nowhere" http://example.com
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	661
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	662 11.2 More on changed methods
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	663
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	664 It should be noted that curl selects which methods to use on its own
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	665 depending on what action to ask for. -d will do POST, -I will do HEAD and so
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	666 on. If you use the --request / -X option you can change the method keyword
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	667 curl selects, but you will not modify curl's behavior. This means that if you
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	668 for example use -d "data" to do a POST, you can modify the method to a
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	669 PROPFIND with -X and curl will still think it sends a POST. You can change
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	670 the normal GET to a POST method by simply adding -X POST in a command line
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	671 like:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	672
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	673 curl -X POST http://example.org/
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	674
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	675 ... but curl will still think and act as if it sent a GET so it won't send any
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	676 request body etc.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	677
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	678
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	679 12. Web Login
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	680
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	681 12.1 Some login tricks
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	682
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	683 While not strictly just HTTP related, it still causes a lot of people problems
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	684 so here's the executive run-down of how the vast majority of all login forms
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	685 work and how to login to them using curl.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	686
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	687 It can also be noted that to do this properly in an automated fashion, you
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	688 will most certainly need to script things and do multiple curl invokes etc.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	689
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	690 First, servers mostly use cookies to track the logged-in status of the
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	691 client, so you will need to capture the cookies you receive in the
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	692 responses. Then, many sites also set a special cookie on the login page (to
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	693 make sure you got there through their login page) so you should make a habit
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	694 of first getting the login-form page to capture the cookies set there.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	695
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	696 Some web-based login systems feature various amounts of javascript, and
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	697 sometimes they use such code to set or modify cookie contents. Possibly they
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	698 do that to prevent programmed logins, like this manual describes how to...
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	699 Anyway, if reading the code isn't enough to let you repeat the behavior
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	700 manually, capturing the HTTP requests done by your browsers and analyzing the
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	701 sent cookies is usually a working method to work out how to shortcut the
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	702 javascript need.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	703
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	704 In the actual <form> tag for the login, lots of sites fill-in random/session
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	705 or otherwise secretly generated hidden tags and you may need to first capture
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	706 the HTML code for the login form and extract all the hidden fields to be able
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	707 to do a proper login POST. Remember that the contents need to be URL encoded
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	708 when sent in a normal POST.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	709
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	710 13. Debug
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	711
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	712 13.1 Some debug tricks
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	713
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	714 Many times when you run curl on a site, you'll notice that the site doesn't
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	715 seem to respond the same way to your curl requests as it does to your
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	716 browser's.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	717
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	718 Then you need to start making your curl requests more similar to your
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	719 browser's requests:
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	720
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	721 * Use the --trace-ascii option to store fully detailed logs of the requests
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	722 for easier analyzing and better understanding
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	723
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	724 * Make sure you check for and use cookies when needed (both reading with
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	725 --cookie and writing with --cookie-jar)
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	726
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	727 * Set user-agent to one like a recent popular browser does
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	728
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	729 * Set referer like it is set by the browser
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	730
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	731 * If you use POST, make sure you send all the fields and in the same order as
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	732 the browser does it.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	733
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	734 A very good helper to make sure you do this right, is the LiveHTTPHeader tool
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	735 that lets you view all headers you send and receive with Mozilla/Firefox
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	736 (even when using HTTPS). Chrome features similar functionality out of the box
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	737 among the developer's tools.
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	738
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	739 A more raw approach is to capture the HTTP traffic on the network with tools
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	740 such as ethereal or tcpdump and check what headers that were sent and
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	741 received by the browser. (HTTPS makes this technique inefficient.)
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	742
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	743 14. References
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	744
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	745 14.1 Standards
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	746
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	747 RFC 7230 is a must to read if you want in-depth understanding of the HTTP
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	748 protocol
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	749
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	750 RFC 3986 explains the URL syntax
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	751
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	752 RFC 1867 defines the HTTP post upload format
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	753
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	754 RFC 6525 defines how HTTP cookies work
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	755
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	756 14.2 Sites
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	757
b50eed0cc0ef ADD: MuPDF v1.26.7: the MuPDF source as downloaded by a default build of PyMuPDF 1.26.4. Franz Glasner <fzglas.hg@dom66.de> parents: diff changeset	758 https://curl.haxx.se is the home of the curl project

Mercurial > hgrepos > Python2 > PyMuPDF

annotate mupdf-source/thirdparty/curl/docs/TheArtOfHttpScripting @ 2:b50eed0cc0ef upstream