Mr Cheng bio photo

Mr Cheng

cURL with cookies

Here’s one way to save a cookie and reuse it with curl:

curl -c $COOKIE_FILE -H "Content-Type: application/json" \
-X POST -d '{"username":"$USERNAME","password":"$PASSWORD"}' \
$LOGIN_URL

curl -b $COOKIE_FILE -H "Content-Type: application/json" \
-X POST -d '{"name":"$NAME"}' \
$BLAH_URL
Beware! The above doesn't seem to work on a Mac (10.11.1) for whatever reason.

more cURL with cookies - curb rubygem

require 'curb'

url = "https://#{hostname}/api/login"
payload = "{\"username\":\"#{username}\",\"password\":\"#{password}\"}"

http = Curl.post(url, payload) do|http|
  http.headers['Content-Type'] = 'application/json'
  http.verbose = true
  http.enable_cookies = true
  http.cookiefile = "./cookiefile"
end
also this seems worthwhile to copy/paste (source: http://blog.codefront.net/2009/06/18/better-cookie-support-in-curb/):
curl = Curl::Easy.new('http://example.com/login')

# Extract cookies in response.
cookies = []
curl.on_header { |header|

  # Parse cookies from the headers (yes, this is a naive implementation but it's fast).
  cookies << "#{$1}=#{$2}" if header =~ /^Set-Cookie: ([^=])=([^;]+;)/

  header.length
}

# POST to login.
curl.http_post(
  Curl::PostField.content('username', 'foo'),
  Curl::PostField.content('password', 'bar')
)

# Reset the on_header handler.
curl.on_header

# Now you can use the auth cookies in future requests.
curl = Curl::Easy.new('http://example.com/private/page')
curl.cookies = cookies
curl.perform

Poking around with Chrome DevTools and cURL

Chrome DevTools lets you peek under the hood, sort of like tcpdump/tshark:

- Fire it up with Option-Command-I.
- Login and poke around the DevTools results.
- Point your browser to anything where you have to login.
- Right-click the failed login request and "Copy as cURL".

Here a form-based login failure to DNSMadeEasy:

curl 'https://cp.dnsmadeeasy.com/login' \
-H 'Cookie: rowNumgridCNAME=250; __utma=154396653.1976802947.1452096368.1452228593.1452228593.1; __utmz=154396653.1452228593.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); ttl=100; _ga=GA1.2.1976802947.1452096368; rowNumgridA=250; JSESSIONID=3066F2FA359BB132179E81A745C979DE' \
-H 'Origin: https://cp.dnsmadeeasy.com' \
-H 'Accept-Encoding: gzip, deflate' \
-H 'Accept-Language: en-US,en;q=0.8' \
-H 'Upgrade-Insecure-Requests: 1' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36' -H 'Content-Type: application/x-www-form-urlencoded' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' \
-H 'Cache-Control: max-age=0' \
-H 'Referer: https://cp.dnsmadeeasy.com/login' \
-H 'Connection: keep-alive' \
--data 'username=wefwef&password=asdfasdf' \
--compressed
It was pretty straight forward - the --data has the username and password, and that's all. The Bank of America --data is more interesting:
curl 'https://secure.bankofamerica.com/login/sign-in/internal/entry/signOnV2.go' \
-H 'Cookie: ___d30306=%7B%22v4a%22%3A%7B%22r%22%3A%220%22%7D%2C%22v4b%22%3A%7B%22f%22%3A%220%22%7D%2C%22v7%22%3A%7B%22s%22%3A%22001A9FE9820056925CFD%22%7D%2C%22v4%22%3A%7B%22j%22%3A%22%22%7D%2C%22timestamp%22%3A%222016-01-10%2013%3A31%3A15%22%2C%22ki%22%3A%221%22%2C%22v6%22%3A%7B%22u%22%3A%22%22%2C%22k%22%3A%22%22%7D%7D%7C%7C8642f849954730257581c0bf40b37491180de3205162baf8b1cca9cbf0ac3d9e; CM_RegCustID=20160107:0:O:146dd5a1-14ef-4289-a4c0821c3f1810a8; NLH=true; PMDATAC=PMV632AedqmtcVecr%2FvLbZB0Z2p6CGX7CBwCyqSOhMDEB5QM3eJ1YoA31IC%2BkCSi9QnqWIGTPZY%2BWSvr7vVfLcV4H8%2Bw%3D%3D; ctd=7CB1400636040C1BC47B03E16B53C8E898E4D1208FE9CE144EC316EAB1FB727C5B0F89EE080E1D4F3B581D6361A6F44937206DE8FFFA5ADF2277A6479A5DDE99; hp_pf_expy=96B2835DF79C29530DFFE4F728B5600E96CCB858ACAA544346002EA4350304702839FCA6CF54FB87E1FD7D78ECA731E6460BD8FC20CEED611F81F1B7BEBEBCC3F9E9DB473397901871B39F1E6758C46B897BB6F62C68A692347CFCF3995AAAC83BD4D8A591B3081E8A4269712792246F2742CD8C72545445C4A6DA9CF09A238AF698787B2DD4780C2C1C9F9903594D26E57C586D8FAC1F116E1386238349690011060AA5B30D40EABC9E48C4F8634A47406EEA1F690EB410F5FEB531F9C2A1E44888ABD345ABC7FD1A9DA34C0EEB73C05826DD9AD2001D23D55CBCE0C32128073E838EC5579F2C301A726B3FC50995EBDDB1FC1B1DA774F554DF5BD61461CF64; TLTSID=55BF667EB79E10B7757DF19942B96418; TLTUID=55BF667EB79E10B7757DF19942B96418; BA_0021=OLB; state=CA; CONTEXT=en_US; INTL_LANG=en_US; LANG_COOKIE=en_US; BOA_0020=20160107:0:O:146dd5a1-14ef-4289-a4c0821c3f1810a8; BOFA_LOCALE_COOKIE=en-US; WPID=F1S1; SID=001A9FE9820056925CFD; LSESSIONID=d939cca50dbc340a269edfa2bd6f3c96593293a6:56925cfe; cmTPSet=Y; SPID=C1S3; JS_VIPAA=00007QE_0u02CjI3cFhU5sP7i1o:18i7ijnhe; mbox=PC#1452197800026-326301.28_70#1460208674|check#true#1452432734|session#1452432638178-367615#1452434534; throttle_value=62; __s30306_5=tdNLV7D0aCPlrdzmj8U%2B3JD5YO9l68FeK0rR9kt4sHF3XWnzT1C8vVO2SUSscWdsKAFKgbktNq%2FcTQaeqBToYg%3D%3D%7C%7CPkNCEo9h7Q8h6K6FV6XHXQ%3D%3D; __s30306_1=AAw7Rly5GLfwwolDWxzWly6urR8wrLk31ne8OlRHsZoKQQmpR%2FCgWWZmBe50lkihs2JwIIGHxuuXAGiYcc4aGw%3D%3D%7C%7C60muq9bJ22DAY%2F2BoZe%2BeQ%3D%3D; __s30306_2=AuXA0uenb232pch4RIqYT5EYjgAqQWUAuGVd1XQvr7g%3D%7C%7CVYbamYdxWlwV1Zr0fzDGrg%3D%3D; __s30306_4=PP55yp5V4WQTChJU7%2FsdR9Ku6tLuNfs1B72nLeNTxJDszYivDZIuOo25cuWbgQ%2B2Wo6lg%2FrZuYm2jf%2Fq9OqI%2F4hJrgqT8YuDQ4TyHDFJRa6wrDInXq3v%2FdrDnMXojfgc%7C%7CEQj%2Bdi10epe5Fd9QsDJD%2Fw%3D%3D; cmRS=&t1=1452432673498&t2=1452432673785&t3=1452432688322&t4=1452432673285&lti=1452432688322&ln=enter-online-id-submit&hr=javascript%3Avoid%280%29%3B&fti=1452432688315&fn=OLBToolSiteKeySign_In_Error_enter-online-id-form%3A0%3B&ac=0:S&fd=0%3A8%3Adummy-onlineId%3B0%3A8%3Adummy-onlineId%3B0%3A11%3Adummy-passcode%3B&uer=&fu=https%3A//secure.bankofamerica.com/login/sign-in/internal/entry/signOnV2.go&pi=OLB%3ATool%3ASiteKey%3BSign_In_Error&ho=sofa.bankofamerica.com/eluminate%3F&ci=90010394' \
-H 'Origin: https://secure.bankofamerica.com' \
-H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.8' \
-H 'Upgrade-Insecure-Requests: 1' \
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' \
-H 'Cache-Control: max-age=0' \
-H 'Referer: https://secure.bankofamerica.com/login/sign-in/signOnV2Screen.go?msg=InvalidCredentialsExceptionV2&request_locale=en-us&lpOlbResetErrorCounter=0' \
-H 'Connection: keep-alive' \
--data 'csrfTokenHidden=b774c1d3b27f327d&f_variable=TF1%3B015%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B%3BMozilla%3BNetscape%3B5.0%2520%2528Macintosh%253B%2520Intel%2520Mac%2520OS%2520X%252010_11_1%2529%2520AppleWebKit%2F537.36%2520%2528KHTML%252C%2520like%2520Gecko%2529%2520Chrome%2F47.0.2526.106%2520Safari%2F537.36%3B20030107%3Bundefined%3Btrue%3B%3Btrue%3BMacIntel%3Bundefined%3BMozilla%2F5.0%2520%2528Macintosh%253B%2520Intel%2520Mac%2520OS%2520X%252010_11_1%2529%2520AppleWebKit%2F537.36%2520%2528KHTML%252C%2520like%2520Gecko%2529%2520Chrome%2F47.0.2526.106%2520Safari%2F537.36%3Ben-US%3Bwindows-1252%3Bsecure.bankofamerica.com%3Bundefined%3Bundefined%3Bundefined%3Bundefined%3Btrue%3Bfalse%3B1452432688314%3B-8%3B6%2F7%2F2005%252C%25209%253A33%253A44%2520PM%3B1440%3B900%3B%3B20.0%3B%3B%3B%3B%3B8%3B480%3B420%3B1%2F10%2F2016%252C%25205%253A31%253A28%2520AM%3B24%3B1440%3B797%3B0%3B23%3B%3B%3B%3B%3B%3BShockwave%2520Flash%257CShockwave%2520Flash%252020.0%2520r0%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B%3B15%3B&lpOlbResetErrorCounter=0&lpPasscodeErrorCounter=0&mouseCapturedEvents=&pm_fp=version%253D1%2526pm%255Ffpua%253Dmozilla%252F5%252E0%2520%2528macintosh%253B%2520intel%2520mac%2520os%2520x%252010%255F11%255F1%2529%2520applewebkit%252F537%252E36%2520%2528khtml%252C%2520like%2520gecko%2529%2520chrome%252F47%252E0%252E2526%252E106%2520safari%252F537%252E36%257C5%252E0%2520%2528Macintosh%253B%2520Intel%2520Mac%2520OS%2520X%252010%255F11%255F1%2529%2520AppleWebKit%252F537%252E36%2520%2528KHTML%252C%2520like%2520Gecko%2529%2520Chrome%252F47%252E0%252E2526%252E106%2520Safari%252F537%252E36%257CMacIntel%2526pm%255Ffpsc%253D24%257C1440%257C900%257C797%2526pm%255Ffpsw%253D%2526pm%255Ffptz%253D%252D8%2526pm%255Ffpln%253Dlang%253Den%252DUS%257Csyslang%253D%257Cuserlang%253D%2526pm%255Ffpjv%253D0%2526pm%255Ffpco%253D1&passcode=asdfwef&onlineId=derpsdfasdf&new-passcode=' \
--compressed
... urldecoding it shows us:
csrfTokenHidden=b774c1d3b27f327d
&f_variable=TF1;015;;;;;;;;;;;;;;;;;;;;;;Mozilla;Netscape;5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36;20030107;undefined;true;;true;MacIntel;undefined;Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36;en-US;windows-1252;secure.bankofamerica.com;undefined;undefined;undefined;undefined;true;false;1452432688314;-8;6/7/2005, 9:33:44 PM;1440;900;;20.0;;;;;8;480;420;1/10/2016, 5:31:28 AM;24;1440;797;0;23;;;;;;Shockwave Flash|Shockwave Flash 20.0 r0;;;;;;;;;;;;;15;
&lpOlbResetErrorCounter=0
&lpPasscodeErrorCounter=0
&mouseCapturedEvents=
&pm_fp=version=1
&pm_fpua=mozilla/5.0 (macintosh; intel mac os x 10_11_1) applewebkit/537.36 (khtml, like gecko) chrome/47.0.2526.106 safari/537.36|5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36|MacIntel
&pm_fpsc=24|1440|900|797
&pm_fpsw=
&pm_fptz=-8
&pm_fpln=lang=en-US|syslang=|userlang=
&pm_fpjv=0
&pm_fpco=1
&passcode=asdfwef
&onlineId=derpsdfasdf
&new-passcode=