327
Clojure世界：Http Client
使用http client提交表單或者下載網頁也是非常常見的任務，比如使用Java的時候可以用標準庫的HttpURLConnection，也可以選擇Apache Http Client。在clojure裏也有這樣的類庫，這裏我將介紹三個各有特色的http client實現。
首先，我最先推薦使用clj-http這個類庫，它是Apache HttpClient的clojure wrapper，是一個提供同步API的簡單易用的Http Client。
名稱: clj-http
主頁：https://github.com/dakrone/clj-http
依賴：
[clj-http "0.3.1"]
例子：
(require '[clj-http.client :as client])
(client/get "https://google.com")
結果：(client/get "https://google.com")
=> ｛:cookies {"NID" {:domain ".google.com.hk", :expires #<Date Tue Aug 14 18:20:38 CST 2012>, :path "/", :value "56=qn2OWtODE2D3fUKi_vbi44jZepOeLI9xC4Ta1JQLEicqUvIZAqr7TCmft_hq8i_FRwnFXdTK1jV2S5IrSZFyYhlAN2KcQEXgWX1iK36gM2iYPaKPihuUZDCqgiAamDOl", :version 0}, "PREF" {:domain ".google.com.hk", :expires #<Date Wed Feb 12 18:20:38 CST 2014>, :path "/", :value "ID=8b73a654ff0a2783:FF=0:NW=1:TM=1329128438:LM=1329128438:S=uEM4SsFuHlkqtVhp", :version 0}},
:status 200
:headers {"date" "Sun, 01 Aug 2010 07:03:49 GMT"
"cache-control" "private, max-age=0"
"content-type" "text/html; charset=ISO-8859-1"
}
:body "<!doctype html>"
:trace-redirects ["https://google.com" "https://www.google.com/" "https://www.google.fr/"]}
更多例子：:status 200
:headers {"date" "Sun, 01 Aug 2010 07:03:49 GMT"
"cache-control" "private, max-age=0"
"content-type" "text/html; charset=ISO-8859-1"
}:body "<!doctype html>
":trace-redirects ["https://google.com" "https://www.google.com/" "https://www.google.fr/"]}
(client/get "https://site.com/resources/3" {:accept :json})
;; Various options:
(client/post "https://site.com/api"
{:basic-auth ["user" "pass"]
:body "{\"json\": \"input\"}"
:headers {"X-Api-Version" "2"}
:content-type :json
:socket-timeout 1000
:conn-timeout 1000
:accept :json})
;; Need to contact a server with an untrusted SSL cert?
(client/get "https://alioth.debian.org" {:insecure? true})
;; If you don't want to follow-redirects automatically:
(client/get "https://site.come/redirects-somewhere" {:follow-redirects false})
;; Only follow a certain number of redirects:
(client/get "https://site.come/redirects-somewhere" {:max-redirects 5})
;; Throw an exception if redirected too many times:
(client/get "https://site.come/redirects-somewhere" {:max-redirects 5 :throw-exceptions true})
;; Send form params as a urlencoded body
(client/post "http//site.com" {:form-params {:foo "bar"}})
;; Multipart form uploads/posts
;; a map or vector works as the multipart object. Use a vector of
;; vectors if you need to preserve order, a map otherwise.
(client/post "http//example.org" {:multipart [["title" "My Awesome Picture"]
["Content/type" "image/jpeg"]
["file" (clojure.java.io/file "pic.jpg")]]})
;; Multipart values can be one of the following:
;; String, InputStream, File, or a byte-array
;; Basic authentication
(client/get "https://site.com/protected" {:basic-auth ["user" "pass"]})
(client/get "https://site.com/protected" {:basic-auth "user:pass"})
;; Query parameters
(client/get "https://site.com/search" {:query-params {"q" "foo, bar"}})
;; Various options:
(client/post "https://site.com/api"
{:basic-auth ["user" "pass"]
:body "{\"json\": \"input\"}"
:headers {"X-Api-Version" "2"}
:content-type :json
:socket-timeout 1000
:conn-timeout 1000
:accept :json})
;; Need to contact a server with an untrusted SSL cert?
(client/get "https://alioth.debian.org" {:insecure? true})
;; If you don't want to follow-redirects automatically:
(client/get "https://site.come/redirects-somewhere" {:follow-redirects false})
;; Only follow a certain number of redirects:
(client/get "https://site.come/redirects-somewhere" {:max-redirects 5})
;; Throw an exception if redirected too many times:
(client/get "https://site.come/redirects-somewhere" {:max-redirects 5 :throw-exceptions true})
;; Send form params as a urlencoded body
(client/post "http//site.com" {:form-params {:foo "bar"}})
;; Multipart form uploads/posts
;; a map or vector works as the multipart object. Use a vector of
;; vectors if you need to preserve order, a map otherwise.
(client/post "http//example.org" {:multipart [["title" "My Awesome Picture"]
["Content/type" "image/jpeg"]
["file" (clojure.java.io/file "pic.jpg")]]})
;; Multipart values can be one of the following:
;; String, InputStream, File, or a byte-array
;; Basic authentication
(client/get "https://site.com/protected" {:basic-auth ["user" "pass"]})
(client/get "https://site.com/protected" {:basic-auth "user:pass"})
;; Query parameters
(client/get "https://site.com/search" {:query-params {"q" "foo, bar"}})
clj-http的API相當的簡潔漂亮，使用起來非常便利，強烈推薦。題外，學習clojure的一個好方法就是為現有的java類庫實現一些方便的clojure wrapper。
如果你需要異步的http client，我會推薦http.async.client這個類庫，它的API是異步形式的類似 Java的Future模式，對於clojure程序員來說應該更像是agent。
名稱：http.async.client
主頁：https://github.com/neotyk/http.async.client
依賴：
[http.async.client "0.4.1"]
例子：
(require '[http.async.client :as c])
(with-open [client (c/create-client)]
(let [response (c/GET client "https://neotyk.github.com/http.async.client/")]
(prn (c/done? response))
(c/await response)
(prn (c/string response))
(prn (c/status response))
(prn (c/done? response))))
(with-open [client (c/create-client)]
(let [response (c/GET client "https://neotyk.github.com/http.async.client/")]
(prn (c/done? response))
(c/await response)
(prn (c/string response))
(prn (c/status response))
(prn (c/done? response))))
輸出：
false
<!DOCTYPE html
{:code 200, :msg "OK", :protocol "HTTP/1.1", :major 1, :minor 1}
true
<!DOCTYPE html
{:code 200, :msg "OK", :protocol "HTTP/1.1", :major 1, :minor 1}
true
更多例子：
(c/POST client "https://example.com" :body "hello world" :timeout 3000)
(c/DELETE client "https://example.com")
(c/POST client "https://example.com" :body "hello world" :auth {:type :basic :user "admin" :password "admin"})
(c/DELETE client "https://example.com")
(c/POST client "https://example.com" :body "hello world" :auth {:type :basic :user "admin" :password "admin"})
請注意，這些方法都是異步調用的，你需要通過await來等待調用完成，或者通過done?來判斷調用是否完成。
http.async.client有個比較重要的特性就是對Http Chunked編碼的支持，分別通過LazySeq和callback的方式支持，首先看將Http chunked變成一個lazy seq:
(with-open [client (client/create-client)] ; Create client
(let [resp (client/stream-seq client :get url)]
(doseq [s (s/string resp)]
(println s))))
(let [resp (client/stream-seq client :get url)]
(doseq [s (s/string resp)]
(println s))))
這裏非常關鍵的一點是stream-seq返回的chunk序列，每取一個就少一個（通過first函數），也就是說每次調用first取到的chunk都不一樣，是順序遞增，不可重複獲取的。
通過callback方式處理：
(with-open [client (client/create-client)] ; Create client
(let [parts (ref #{})
resp (client/request-stream client :get url
(fn [state body]
(dosync (alter parts conj (string body)))
[body :continue]))]
;; do something to @parts
))
(let [parts (ref #{})
resp (client/request-stream client :get url
(fn [state body]
(dosync (alter parts conj (string body)))
[body :continue]))]
;; do something to @parts
))
自己傳入一個callback函數接收chunk，比如這裏用一個ref累積。
http.async.client的詳細文檔看這裏：https://neotyk.github.com/http.async.client/docs.html
最後，有興趣還可以看下aleph這個異步通訊的框架，它支持Http協議，也提供了http server和client的實現。不過它的API就沒有那麼簡單明了，它的模型是類似go語言裏利用channel做異步通訊的模型，http隻是它的一個模塊罷了，這是另一個話題了。
文章轉自莊周夢蝶 ，原文發布時間2012-02-13
最後更新：2017-05-18 20:36:08