Page MenuHomePhorge

No OneTemporary

Size
46 KB
Referenced Files
None
Subscribers
None
diff --git a/README.md b/README.md
index 5372d4a..394bf5e 100644
--- a/README.md
+++ b/README.md
@@ -1,499 +1,499 @@
# Tesla
[![Build Status](https://github.com/teamon/tesla/workflows/Test/badge.svg)](https://github.com/teamon/tesla/actions)
[![Hex.pm](https://img.shields.io/hexpm/v/tesla.svg)](http://hex.pm/packages/tesla)
[![Hex.pm](https://img.shields.io/hexpm/dt/tesla.svg)](https://hex.pm/packages/tesla)
[![Hex.pm](https://img.shields.io/hexpm/dw/tesla.svg)](https://hex.pm/packages/tesla)
[![codecov](https://codecov.io/gh/teamon/tesla/branch/master/graph/badge.svg)](https://codecov.io/gh/teamon/tesla)
[![Inline docs](https://inch-ci.org/github/teamon/tesla.svg)](http://inch-ci.org/github/teamon/tesla)
Tesla is an HTTP client loosely based on [Faraday](https://github.com/lostisland/faraday).
It embraces the concept of middleware when processing the request/response cycle.
> Note that this README refers to the `master` branch of Tesla, not the latest
released version on Hex. See [the documentation](http://hexdocs.pm/tesla) for
the documentation of the version you're using.
<hr/>
## [`0.x` to `1.0` Migration Guide](https://github.com/teamon/tesla/wiki/0.x-to-1.0-Migration-Guide)
```elixir
defp deps do
[{:tesla, "~> 1.3.0"}]
end
```
[Documentation for 0.x branch](https://github.com/teamon/tesla/tree/0.x)
<hr/>
## HTTP Client example
Define module with `use Tesla` and choose from a variety of middleware.
```elixir
defmodule GitHub do
use Tesla
plug Tesla.Middleware.BaseUrl, "https://api.github.com"
plug Tesla.Middleware.Headers, [{"authorization", "token xyz"}]
plug Tesla.Middleware.JSON
def user_repos(login) do
get("/user/" <> login <> "/repos")
end
end
```
Then use it like this:
```elixir
{:ok, response} = GitHub.user_repos("teamon")
response.status # => 200
response.body # => [%{…}, …]
response.headers # => [{"content-type", "application/json"}, ...]
```
See below for documentation.
## Installation
Add `tesla` as dependency in `mix.exs`:
```elixir
defp deps do
[
{:tesla, "~> 1.3.0"},
{:hackney, "~> 1.14.0"}, # optional, but recommended adapter
{:jason, ">= 1.0.0"} # optional, required by JSON middleware
- ]
+ ]
end
```
Configure default adapter in `config/config.exs` (optional).
```elixir
# config/config.exs
config :tesla, adapter: Tesla.Adapter.Hackney
```
> The default adapter is erlang's built-in `httpc`, but it is not recommended
to use it in production environment as it does not validate SSL certificates
[among other issues](https://github.com/teamon/tesla/issues?utf8=%E2%9C%93&q=is%3Aissue+label%3Ahttpc+).
## Documentation
- [Middleware](#middleware)
- [Runtime middleware](#runtime-middleware)
- [Adapters](#adapters)
- [Streaming](#streaming)
- [Multipart](#multipart)
- [Testing](#testing)
- [Writing middleware](#writing-middleware)
- [Direct usage](#direct-usage)
- [Cheatsheet](#cheatsheet)
- [Cookbook](https://github.com/teamon/tesla/wiki)
- [Changelog](https://github.com/teamon/tesla/releases)
## Middleware
Tesla is built around the concept of composable middlewares.
This is very similar to how [Plug Router](https://github.com/elixir-plug/plug#the-plug-router) works.
#### Basic
- [`Tesla.Middleware.BaseUrl`](https://hexdocs.pm/tesla/Tesla.Middleware.BaseUrl.html) - set base url
- [`Tesla.Middleware.Headers`](https://hexdocs.pm/tesla/Tesla.Middleware.Headers.html) - set request headers
- [`Tesla.Middleware.Query`](https://hexdocs.pm/tesla/Tesla.Middleware.Query.html) - set query parameters
- [`Tesla.Middleware.Opts`](https://hexdocs.pm/tesla/Tesla.Middleware.Opts.html) - set request options
- [`Tesla.Middleware.FollowRedirects`](https://hexdocs.pm/tesla/Tesla.Middleware.FollowRedirects.html) - follow 3xx redirects
- [`Tesla.Middleware.MethodOverride`](https://hexdocs.pm/tesla/Tesla.Middleware.MethodOverride.html) - set X-Http-Method-Override
- [`Tesla.Middleware.Logger`](https://hexdocs.pm/tesla/Tesla.Middleware.Logger.html) - log requests (method, url, status, time)
- [`Tesla.Middleware.KeepRequest`](https://hexdocs.pm/tesla/Tesla.Middleware.KeepRequest.html) - keep request body & headers
- [`Tesla.Middleware.PathParams`](https://hexdocs.pm/tesla/Tesla.Middleware.PathParams.html) - use templated URLs
#### Formats
- [`Tesla.Middleware.FormUrlencoded`](https://hexdocs.pm/tesla/Tesla.Middleware.FormUrlencoded.html) - urlencode POST body parameter, useful for POSTing a map/keyword list
- [`Tesla.Middleware.JSON`](https://hexdocs.pm/tesla/Tesla.Middleware.JSON.html) - JSON request/response body
- [`Tesla.Middleware.Compression`](https://hexdocs.pm/tesla/Tesla.Middleware.Compression.html) - gzip & deflate
- [`Tesla.Middleware.DecodeRels`](https://hexdocs.pm/tesla/Tesla.Middleware.DecodeRels.html) - decode `Link` header into `opts[:rels]` field in response
#### Auth
- [`Tesla.Middleware.BasicAuth`](https://hexdocs.pm/tesla/Tesla.Middleware.BasicAuth.html) - HTTP Basic Auth
- [`Tesla.Middleware.DigestAuth`](https://hexdocs.pm/tesla/Tesla.Middleware.DigestAuth.html) - Digest access authentication
#### Error handling
- [`Tesla.Middleware.Timeout`](https://hexdocs.pm/tesla/Tesla.Middleware.Timeout.html) - timeout request after X milliseconds despite of server response
- [`Tesla.Middleware.Retry`](https://hexdocs.pm/tesla/Tesla.Middleware.Retry.html) - retry few times in case of connection refused
- [`Tesla.Middleware.Fuse`](https://hexdocs.pm/tesla/Tesla.Middleware.Fuse.html) - fuse circuit breaker integration
## Runtime middleware
All HTTP functions (`get`, `post`, etc.) can take a dynamic client as the first parameter.
This allow to use convenient syntax for modifying the behaviour in runtime.
Consider the following case: GitHub API can be accessed using OAuth token authorization.
We can't use `plug Tesla.Middleware.Headers, [{"authorization", "token here"}]`
since this would be compiled only once and there is no way to insert dynamic user token.
Instead, we can use `Tesla.client` to create a client with dynamic middleware:
```elixir
defmodule GitHub do
# notice there is no `use Tesla`
def user_repos(client, login) do
# pass `client` argument to `Tesla.get` function
Tesla.get(client, "/user/" <> login <> "/repos")
end
def issues(client) do
Tesla.get(client, "/issues")
end
# build dynamic client based on runtime arguments
def client(token) do
middleware = [
{Tesla.Middleware.BaseUrl, "https://api.github.com"},
Tesla.Middleware.JSON,
{Tesla.Middleware.Headers, [{"authorization", "token: " <> token }]}
]
Tesla.client(middleware)
end
end
```
and then:
```elixir
client = GitHub.client(user_token)
client |> GitHub.user_repos("teamon")
client |> GitHub.get("/me")
```
## Adapters
Tesla supports multiple HTTP adapter that do the actual HTTP request processing.
- [`Tesla.Adapter.Httpc`](https://hexdocs.pm/tesla/Tesla.Adapter.Httpc.html) - the default, built-in erlang [httpc](http://erlang.org/doc/man/httpc.html) adapter
- [`Tesla.Adapter.Hackney`](https://hexdocs.pm/tesla/Tesla.Adapter.Hackney.html) - [hackney](https://github.com/benoitc/hackney), "simple HTTP client in Erlang"
- [`Tesla.Adapter.Ibrowse`](https://hexdocs.pm/tesla/Tesla.Adapter.Ibrowse.html) - [ibrowse](https://github.com/cmullaparthi/ibrowse), "Erlang HTTP client"
-- [`Tesla.Adapter.Gun`](https://hexdocs.pm/tesla/Tesla.Adapter.Gun.html) - [gun](https://github.com/ninenines/gun), "HTTP/1.1, HTTP/2 and Websocket client for Erlang/OTP."
+- [`Tesla.Adapter.Gun`](https://hexdocs.pm/tesla/Tesla.Adapter.Gun.html) - [gun](https://github.com/ninenines/gun), "HTTP/1.1, HTTP/2 and Websocket client for Erlang/OTP"
- [`Tesla.Adapter.Mint`](https://hexdocs.pm/tesla/Tesla.Adapter.Mint.html) - [mint](https://github.com/ericmj/mint), "Functional HTTP client for Elixir with support for HTTP/1 and HTTP/2"
When using adapter other than httpc remember to add it to the dependencies list in `mix.exs`
```elixir
defp deps do
[{:tesla, "~> 1.3.0"},
{:jason, ">= 1.0.0"}, # optional, required by JSON middleware
{:hackney, "~> 1.10"}] # or :gun etc.
end
```
### Adapter options
In case there is a need to pass specific adapter options you can do it in one of three ways:
Using `adapter` macro:
```elixir
defmodule GitHub do
use Tesla
adapter Tesla.Adapter.Hackney, recv_timeout: 30_000, ssl_options: [certfile: "certs/client.crt"]
end
```
Using `Tesla.client/2`:
```elixir
def new(...) do
middleware = [...]
adapter = {Tesla.Adapter.Hackney, [recv_timeout: 30_000]}
Tesla.client(middleware, adapter)
end
```
Passing directly to `get`/`post`/etc.
```elixir
MyClient.get("/", opts: [adapter: [recv_timeout: 30_000]])
Tesla.get(client, "/", opts: [adapter: [recv_timeout: 30_000]])
```
## Streaming
If adapter supports it, you can pass a [Stream](http://elixir-lang.org/docs/stable/elixir/Stream.html) as body, e.g.:
```elixir
defmodule ElasticSearch do
use Tesla
plug Tesla.Middleware.BaseUrl, "http://localhost:9200"
plug Tesla.Middleware.JSON
def index(records_stream) do
stream = records_stream |> Stream.map(fn record -> %{index: [some, data]} end)
post("/_bulk", stream)
end
end
```
Each piece of stream will be encoded as JSON and sent as a new line (conforming to JSON stream format)
## Multipart
You can pass a `Tesla.Multipart` struct as the body.
```elixir
alias Tesla.Multipart
mp =
Multipart.new
|> Multipart.add_content_type_param("charset=utf-8")
|> Multipart.add_field("field1", "foo")
|> Multipart.add_field("field2", "bar", headers: [{"content-id", "1"}, {"content-type", "text/plain"}])
|> Multipart.add_file("test/tesla/multipart_test_file.sh")
|> Multipart.add_file("test/tesla/multipart_test_file.sh", name: "foobar")
|> Multipart.add_file_content("sample file content", "sample.txt")
{:ok, response} = MyApiClient.post("http://httpbin.org/post", mp)
```
## Testing
You can set the adapter to `Tesla.Mock` in tests.
```elixir
# config/test.exs
# Use mock adapter for all clients
config :tesla, adapter: Tesla.Mock
# or only for one
config :tesla, MyApi, adapter: Tesla.Mock
```
Then, mock requests before using your client:
```elixir
defmodule MyAppTest do
use ExUnit.Case
import Tesla.Mock
setup do
mock fn
%{method: :get, url: "http://example.com/hello"} ->
%Tesla.Env{status: 200, body: "hello"}
%{method: :post, url: "http://example.com/world"} ->
json(%{"my" => "data"})
end
:ok
end
test "list things" do
assert {:ok, %Tesla.Env{} = env} = MyApp.get("/hello")
assert env.status == 200
assert env.body == "hello"
end
end
```
## Writing middleware
A Tesla middleware is a module with `call/3` function, that at some point calls `Tesla.run(env, next)` to process
the rest of stack.
```elixir
defmodule MyMiddleware do
@behaviour Tesla.Middleware
def call(env, next, options) do
env
|> do_something_with_request
|> Tesla.run(next)
|> do_something_with_response
end
end
```
The arguments are:
- `env` - `Tesla.Env` instance
- `next` - middleware continuation stack; to be executed with `Tesla.run(env, next)`
- `options` - arguments passed during middleware configuration (`plug MyMiddleware, options`)
There is no distinction between request and response middleware, it's all about executing `Tesla.run/2` function at the correct time.
For example, a request logger middleware could be implemented like this:
```elixir
defmodule Tesla.Middleware.RequestLogger do
@behaviour Tesla.Middleware
def call(env, next, _) do
IO.inspect env # print request env
Tesla.run(env, next)
end
end
```
and response logger middleware like this:
```elixir
defmodule Tesla.Middleware.ResponseLogger do
@behaviour Tesla.Middleware
def call(env, next, _) do
res = Tesla.run(env, next)
IO.inspect res # print response env
res
end
end
```
See [built-in middlewares](https://github.com/teamon/tesla/tree/master/lib/tesla/middleware) for more examples.
Middleware should have documentation following this template:
````elixir
defmodule Tesla.Middleware.SomeMiddleware do
@behaviour Tesla.Middleware
@moduledoc """
Short description what it does
Longer description, including e.g. additional dependencies.
### Example usage
```
defmodule MyClient do
use Tesla
plug Tesla.Middleware.SomeMiddleware, most: :common, options: "here"
end
```
### Options
- `:list` - all possible options
- `:with` - their default values
"""
end
````
## Direct usage
You can also use Tesla directly, without creating a client module.
This however won’t include any middleware.
```elixir
# Example get request
{:ok, response} = Tesla.get("http://httpbin.org/ip")
response.status # => 200
response.body # => "{\n "origin": "87.205.72.203"\n}\n"
response.headers # => [{"content-type", "application/json" ...}]
{:ok, response} = Tesla.get("http://httpbin.org/get", query: [a: 1, b: "foo"])
response.url # => "http://httpbin.org/get?a=1&b=foo"
# Example post request
{:ok, response} = Tesla.post("http://httpbin.org/post", "data", headers: [{"content-type", "application/json"}])
```
## Cheatsheet
#### Making requests 101
```elixir
# GET /path
get("/path")
# GET /path?a=hi&b[]=1&b[]=2&b[]=3
get("/path", query: [a: "hi", b: [1,2,3]])
# GET with dynamic client
get(client, "/path")
get(client, "/path", query: [page: 3])
# arguments are the same for GET, HEAD, OPTIONS & TRACE
head("/path")
options("/path")
trace("/path")
# POST, PUT, PATCH
post("/path", "some-body-i-used-to-know")
put("/path", "some-body-i-used-to-know", query: [a: "0"])
patch("/path", multipart)
```
#### Configuring HTTP functions visibility
```elixir
# generate only get and post function
use Tesla, only: ~w(get post)a
# generate only delete fuction
use Tesla, only: [:delete]
# generate all functions except delete and options
use Tesla, except: [:delete, :options]
```
#### Disable docs for HTTP functions
```elixir
use Tesla, docs: false
```
#### Decode only JSON response (do not encode request)
```elixir
plug Tesla.Middleware.DecodeJson
```
#### Use other JSON library
```elixir
# use JSX
plug Tesla.Middleware.JSON, engine: JSX, engine_opts: [strict: [:comments]]
# use custom functions
plug Tesla.Middleware.JSON, decode: &JSX.decode/1, encode: &JSX.encode/1
```
#### Custom middleware
```elixir
defmodule Tesla.Middleware.MyCustomMiddleware do
def call(env, next, options) do
env
|> do_something_with_request
|> Tesla.run(next)
|> do_something_with_response
end
end
```
## Contributing
1. Fork it (https://github.com/teamon/tesla/fork)
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create new Pull Request
## License
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details
Copyright (c) 2015-2018 [Tymon Tobolski](http://teamon.eu/about/)
diff --git a/lib/tesla/adapter/gun.ex b/lib/tesla/adapter/gun.ex
index 10177c7..fe8b0e5 100644
--- a/lib/tesla/adapter/gun.ex
+++ b/lib/tesla/adapter/gun.ex
@@ -1,381 +1,422 @@
if Code.ensure_loaded?(:gun) do
defmodule Tesla.Adapter.Gun do
@moduledoc """
Adapter for [gun](https://github.com/ninenines/gun)
-
Remember to add `{:gun, "~> 1.3"}` to dependencies.
In version 1.3 gun sends `host` header with port. Fixed in master branch.
Also, you need to recompile tesla after adding `:gun` dependency:
```
mix deps.clean tesla
mix deps.compile tesla
```
### Example usage
```
# set globally in config/config.exs
config :tesla, :adapter, Tesla.Adapter.Gun
# set per module
defmodule MyClient do
use Tesla
adapter Tesla.Adapter.Gun
end
```
-
### Adapter specific options:
-
* `timeout` - Time, while process, will wait for gun messages.
* `body_as` - What will be returned in `%Tesla.Env{}` body key. Possible values - `:plain`, `:stream`, `:chunks`. Defaults to `:plain`.
* `:plain` - as binary.
* `:stream` - as stream. If you don't want to close connection (because you want to reuse it later) pass `close_conn: false` in adapter opts.
* `:chunks` - as chunks. You can get response body in chunks using `Tesla.Adapter.Gun.read_chunk/3` function.
-
- Processing of the chunks and checking body size must be done by yourself. Example of processing function is in `test/tesla/adapter/gun_test.exs` - `read_body/3`. If you don't need connection later don't forget to close it with `Tesla.Adapter.Gun.close/1`.
+ Processing of the chunks and checking body size must be done by yourself. Example of processing function is in `test/tesla/adapter/gun_test.exs` - `Tesla.Adapter.GunTest.read_body/4`. If you don't need connection later don't forget to close it with `Tesla.Adapter.Gun.close/1`.
* `max_body` - Max response body size in bytes. Works only with `body_as: :plain`, with other settings you need to check response body size by yourself.
* `conn` - Opened connection pid with gun. Is used for reusing gun connections.
- * `original` - Original host with port, for which reused connection was open. Needed for `Tesla.Middleware.FollowRedirects`. Otherwise adapter will use connection for another open host.
+ * `original` - Original host with port, for which reused connection was open. Needed for `Tesla.Middleware.FollowRedirects`. Otherwise adapter will use connection for another open host. Example: `"example.com:80"`.
* `close_conn` - Close connection or not after receiving full response body. Is used for reusing gun connections. Defaults to `true`.
* `certificates_verification` - Add SSL certificates verification. [erlang-certifi](https://github.com/certifi/erlang-certifi) [ssl_verify_fun.erl](https://github.com/deadtrickster/ssl_verify_fun.erl)
-
+ * `proxy` - Proxy for requests. **Socks proxy are supported only for gun master branch**. Examples: `{'localhost', 1234}`, `{{127, 0, 0, 1}, 1234}`, `{:socks5, 'localhost', 1234}`.
### [Gun options](https://ninenines.eu/docs/en/gun/1.3/manual/gun/):
-
* `connect_timeout` - Connection timeout.
* `http_opts` - Options specific to the HTTP protocol.
* `http2_opts` - Options specific to the HTTP/2 protocol.
* `protocols` - Ordered list of preferred protocols. Defaults: [http2, http] - for :tls, [http] - for :tcp.
* `trace` - Whether to enable dbg tracing of the connection process. Should only be used during debugging. Default: false.
* `transport` - Whether to use TLS or plain TCP. The default varies depending on the port used. Port 443 defaults to tls. All other ports default to tcp.
* `transport_opts` - Transport options. They are TCP options or TLS options depending on the selected transport. Default: []. Gun version: 1.3
* `tls_opts` - TLS transport options. Default: []. Gun from master branch.
* `tcp_opts` - TCP trasnport options. Default: []. Gun from master branch.
+ * `socks_opts` - Options for socks. Default: []. Gun from master branch.
* `ws_opts` - Options specific to the Websocket protocol. Default: %{}.
* `compress` - Whether to enable permessage-deflate compression. This does not guarantee that compression will be used as it is the server that ultimately decides. Defaults to false.
* `protocols` - A non-empty list enables Websocket protocol negotiation. The list of protocols will be sent in the sec-websocket-protocol request header. The handler module interface is currently undocumented and must be set to `gun_ws_h`.
-
"""
@behaviour Tesla.Adapter
alias Tesla.Multipart
@gun_keys [
:connect_timeout,
:http_opts,
:http2_opts,
:protocols,
:retry,
:retry_timeout,
:trace,
:transport,
+ :socks_opts,
:ws_opts
]
@adapter_default_timeout 1_000
@impl true
@doc false
def call(env, opts) do
with {:ok, status, headers, body} <- request(env, opts) do
{:ok, %{env | status: status, headers: format_headers(headers), body: body}}
end
end
+ @doc """
+ Reads chunk of the response body.
+ Returns `{:fin, binary()}` if all body received, otherwise returns `{:nofin, binary()}`.
+ """
+ @spec read_chunk(pid(), reference(), keyword() | map()) ::
+ {:fin, binary()} | {:nofin, binary()} | {:error, :timeout}
+ def read_chunk(pid, stream, opts) do
+ receive do
+ {:gun_data, ^pid, ^stream, :fin, body} ->
+ if opts[:close_conn], do: close(pid)
+ {:fin, body}
+
+ {:gun_data, ^pid, ^stream, :nofin, part} ->
+ {:nofin, part}
+
+ {:DOWN, _, _, _, reason} ->
+ if opts[:close_conn], do: close(pid)
+ {:error, reason}
+ after
+ opts[:timeout] || @adapter_default_timeout ->
+ {:error, :recv_chunk_timeout}
+ end
+ end
+
+ @doc """
+ Brutally close the `gun` connection
+ """
+ @spec close(pid()) :: :ok
+ defdelegate close(pid), to: :gun
+
defp format_headers(headers) do
for {key, value} <- headers do
{String.downcase(to_string(key)), to_string(value)}
end
end
- defp format_method(method), do: String.upcase(to_string(method))
-
- defp format_url(nil, nil), do: "/"
- defp format_url(nil, query), do: "/?" <> query
- defp format_url(path, nil), do: path
- defp format_url(path, query), do: path <> "?" <> query
-
defp request(env, opts) do
request(
- format_method(env.method),
+ Tesla.Adapter.Shared.format_method(env.method),
Tesla.build_url(env.url, env.query),
- env.headers,
+ format_headers(env.headers),
env.body || "",
Tesla.Adapter.opts(
[close_conn: true, body_as: :plain, send_body: :at_once, receive: true],
env,
opts
)
|> Enum.into(%{})
)
end
defp request(method, url, headers, %Stream{} = body, opts),
do: request_stream(method, url, headers, body, Map.put(opts, :send_body, :stream))
defp request(method, url, headers, body, opts) when is_function(body),
do: request_stream(method, url, headers, body, Map.put(opts, :send_body, :stream))
defp request(method, url, headers, %Multipart{} = mp, opts) do
headers = headers ++ Multipart.headers(mp)
body = Multipart.body(mp)
request(method, url, headers, body, opts)
end
defp request(method, url, headers, body, opts),
do: do_request(method, url, headers, body, opts)
defp request_stream(method, url, headers, body, opts),
do: do_request(method, url, headers, body, opts)
defp do_request(method, url, headers, body, opts) do
with uri <- URI.parse(url),
- path_with_query <- format_url(uri.path, uri.query),
- {pid, opts} <- open_conn(uri, opts),
- stream <- open_stream(pid, method, path_with_query, headers, body, opts[:send_body]) do
+ path <- Tesla.Adapter.Shared.prepare_path(uri.path, uri.query),
+ opts <- check_original(uri, opts),
+ {:ok, pid, opts} <- open_conn(uri, opts),
+ stream <- open_stream(pid, method, path, headers, body, opts) do
read_response(pid, stream, opts)
end
end
- defp open_conn(uri, %{conn: conn, original: original} = opts) do
- if original == "#{uri.host}:#{uri.port}" do
- {conn, Map.put(opts, :receive, false)}
- else
- # current url is different from the original, maybe there were redirects
- # so we can't use transferred connection
- open_conn(uri, Map.delete(opts, :conn))
- end
+ defp check_original(uri, %{original: original} = opts) do
+ Map.put(opts, :original_matches, original == "#{uri.host}:#{uri.port}")
+ end
+
+ defp check_original(_uri, opts), do: opts
+
+ defp open_conn(_uri, %{conn: conn, original_matches: true} = opts) do
+ {:ok, conn, Map.put(opts, :receive, false)}
+ end
+
+ defp open_conn(uri, %{conn: conn, original_matches: false} = opts) do
+ # current url is different from the original, so we can't use transferred connection
+ opts =
+ opts
+ |> Map.put_new(:old_conn, conn)
+ |> Map.delete(:conn)
+
+ open_conn(uri, opts)
end
defp open_conn(uri, opts) do
opts =
if uri.scheme == "https" and uri.port != 443 do
Map.put(opts, :transport, :tls)
else
opts
end
- # We need to add `server_name_indication` option, because gun connects through ip in master branch.
- # [SNI] - http://erlang.org/doc/man/ssl.html#type-sni
- # Support for gun master branch where transport_opts, were splitted to tls_opts and tcp_opts
- # https://github.com/ninenines/gun/blob/491ddf58c0e14824a741852fdc522b390b306ae2/doc/src/manual/gun.asciidoc#changelog
-
tls_opts =
Map.get(opts, :tls_opts, [])
|> Keyword.merge(Map.get(opts, :transport_opts, []))
tls_opts =
- if uri.scheme == "https" do
- host = uri.host |> to_charlist()
+ with "https" <- uri.scheme,
+ false <- opts[:original_matches] do
+ # current url is different from the original, so we can't use verify_fun for https requests
+ Keyword.delete(tls_opts, :verify_fun)
+ else
+ _ -> tls_opts
+ end
+
+ # Support for gun master branch where transport_opts, were splitted to tls_opts and tcp_opts
+ # https://github.com/ninenines/gun/blob/491ddf58c0e14824a741852fdc522b390b306ae2/doc/src/manual/gun.asciidoc#changelog
- ssl_opts = [
- server_name_indication: host
+ tls_opts =
+ with "https" <- uri.scheme,
+ true <- opts[:certificates_verification],
+ host <- to_charlist(uri.host) do
+ host = :idna.encode(host)
+
+ security_opts = [
+ verify: :verify_peer,
+ cacertfile: CAStore.file_path(),
+ depth: 20,
+ reuse_sessions: false,
+ verify_fun: {&:ssl_verify_hostname.verify_fun/3, [check_hostname: host]}
]
- ssl_opts =
- if opts[:certificates_verification] do
- security_opts = [
- verify: :verify_peer,
- cacerts: :certifi.cacerts(),
- depth: 20,
- reuse_sessions: false,
- verify_fun: {&:ssl_verify_hostname.verify_fun/3, [check_hostname: host]}
- ]
-
- Keyword.merge(ssl_opts, security_opts)
- else
- ssl_opts
- end
-
- Keyword.merge(tls_opts, ssl_opts)
+ Keyword.merge(security_opts, tls_opts)
else
- tls_opts
+ _ -> tls_opts
end
gun_opts = Map.take(opts, @gun_keys)
with {:ok, pid} <- do_open_conn(uri, opts, gun_opts, tls_opts) do
# If there were redirects, and passed `closed_conn: false`, we need to close opened connections to these intermediate hosts.
- {pid, Map.put(opts, :close_conn, true)}
+ {:ok, pid, Map.put(opts, :close_conn, true)}
end
end
defp do_open_conn(uri, %{proxy: {proxy_host, proxy_port}}, gun_opts, tls_opts) do
- connect_opts = %{host: to_charlist(uri.host), port: uri.port}
-
connect_opts =
- if uri.scheme == "https" do
- Map.put(connect_opts, :protocols, [:http2])
- |> Map.put(:transport, :tls)
- |> Map.put(:tls_opts, tls_opts)
- else
- connect_opts
- end
+ tunnel_opts(uri)
+ |> tunnel_tls_opts(uri.scheme, tls_opts)
with {:ok, pid} <- :gun.open(proxy_host, proxy_port, gun_opts),
{:ok, _} <- :gun.await_up(pid),
stream <- :gun.connect(pid, connect_opts),
{:response, :fin, 200, _} <- :gun.await(pid, stream) do
{:ok, pid}
end
end
+ defp do_open_conn(uri, %{proxy: {proxy_type, proxy_host, proxy_port}}, gun_opts, tls_opts) do
+ received_version =
+ to_string(proxy_type)
+ |> String.last()
+
+ version =
+ if received_version in ["4", "5"] do
+ String.to_integer(received_version)
+ else
+ 5
+ end
+
+ socks_opts =
+ tunnel_opts(uri)
+ |> tunnel_tls_opts(uri.scheme, tls_opts)
+ |> Map.put(:version, version)
+
+ gun_opts =
+ Map.put(gun_opts, :protocols, [:socks])
+ |> Map.update(:socks_opts, socks_opts, &Map.merge(socks_opts, &1))
+
+ with {:ok, pid} <- :gun.open(proxy_host, proxy_port, gun_opts),
+ {:ok, _} <- :gun.await_up(pid) do
+ {:ok, pid}
+ else
+ {:error, {:options, {:protocols, [:socks]}}} ->
+ {:error, "socks protocol is not supported"}
+ end
+ end
+
defp do_open_conn(uri, opts, gun_opts, tls_opts) do
tcp_opts = Map.get(opts, :tcp_opts, [])
# if gun used from master
opts_with_master_keys =
Map.put(gun_opts, :tls_opts, tls_opts)
|> Map.put(:tcp_opts, tcp_opts)
- host = to_charlist(uri.host)
+ host =
+ uri.host
+ |> to_charlist()
+ |> :idna.encode()
with {:ok, pid} <- :gun.open(host, uri.port, opts_with_master_keys) do
{:ok, pid}
else
{:error, {:options, {key, _}}} when key in [:tcp_opts, :tls_opts] ->
:gun.open(
host,
uri.port,
Map.put(gun_opts, :transport_opts, tls_opts)
)
end
end
- defp open_stream(pid, method, url, headers, body, :stream) do
- stream = :gun.request(pid, method, url, headers, "")
+ defp tunnel_opts(uri), do: %{host: :idna.encode(to_charlist(uri.host)), port: uri.port}
+
+ defp tunnel_tls_opts(opts, "https", tls_opts) do
+ http2_opts = %{protocols: [:http2], transport: :tls, tls_opts: tls_opts}
+ Map.merge(opts, http2_opts)
+ end
+
+ defp tunnel_tls_opts(opts, _, _), do: opts
+
+ defp open_stream(pid, method, path, headers, body, opts) do
+ req_opts = %{reply_to: opts[:reply_to] || self()}
+
+ open_stream(pid, method, path, headers, body, req_opts, opts[:send_body])
+ end
+
+ defp open_stream(pid, method, path, headers, body, req_opts, :stream) do
+ stream = :gun.request(pid, method, path, headers, "", req_opts)
for data <- body, do: :ok = :gun.data(pid, stream, :nofin, data)
:gun.data(pid, stream, :fin, "")
stream
end
- defp open_stream(pid, method, url, headers, body, :at_once),
- do: :gun.request(pid, method, url, headers, body)
+ defp open_stream(pid, method, path, headers, body, req_opts, :at_once),
+ do: :gun.request(pid, method, path, headers, body, req_opts)
defp read_response(pid, stream, opts) do
receive? = opts[:receive]
receive do
{:gun_response, ^pid, ^stream, :fin, status, headers} ->
if opts[:close_conn], do: close(pid)
{:ok, status, headers, ""}
{:gun_response, ^pid, ^stream, :nofin, status, headers} ->
format_response(pid, stream, opts, status, headers, opts[:body_as])
- {:error, error} ->
- if opts[:close_conn], do: close(pid)
- {:error, error}
-
{:gun_up, ^pid, _protocol} when receive? ->
read_response(pid, stream, opts)
{:gun_error, ^pid, reason} ->
if opts[:close_conn], do: close(pid)
{:error, reason}
{:gun_down, ^pid, _, _, _, _} when receive? ->
read_response(pid, stream, opts)
{:DOWN, _, _, _, reason} ->
if opts[:close_conn], do: close(pid)
{:error, reason}
after
opts[:timeout] || @adapter_default_timeout ->
if opts[:close_conn], do: :ok = close(pid)
- {:error, :timeout}
+ {:error, :recv_response_timeout}
end
end
defp format_response(pid, stream, opts, status, headers, :plain) do
case read_body(pid, stream, opts) do
{:ok, body} ->
if opts[:close_conn], do: close(pid)
{:ok, status, headers, body}
{:error, error} ->
if opts[:close_conn], do: close(pid)
{:error, error}
end
end
defp format_response(pid, stream, opts, status, headers, :stream) do
stream_body =
Stream.resource(
fn -> %{pid: pid, stream: stream} end,
fn
%{pid: pid, stream: stream} ->
case read_chunk(pid, stream, opts) do
{:nofin, part} -> {[part], %{pid: pid, stream: stream}}
{:fin, body} -> {[body], %{pid: pid, final: :fin}}
end
%{pid: pid, final: :fin} ->
{:halt, %{pid: pid}}
end,
fn %{pid: pid} ->
if opts[:close_conn], do: close(pid)
end
)
{:ok, status, headers, stream_body}
end
defp format_response(pid, stream, opts, status, headers, :chunks) do
{:ok, status, headers, %{pid: pid, stream: stream, opts: Enum.into(opts, [])}}
end
- @doc """
- Reads chunk of the response body.
- Returns `{:fin, binary()}` if all body received, otherwise returns `{:nofin, binary()}`.
- """
- @spec read_chunk(pid(), reference(), keyword() | map()) ::
- {:fin, binary()} | {:nofin, binary()} | {:error, :timeout}
- def read_chunk(pid, stream, opts) do
- receive do
- {:gun_data, ^pid, ^stream, :fin, body} ->
- {:fin, body}
-
- {:gun_data, ^pid, ^stream, :nofin, part} ->
- {:nofin, part}
- after
- opts[:timeout] || @adapter_default_timeout ->
- {:error, :timeout}
- end
- end
-
- @doc """
- Brutally close the `gun` connection
- """
- @spec close(pid()) :: :ok
- def close(pid) do
- :gun.close(pid)
- end
-
defp read_body(pid, stream, opts, acc \\ "") do
limit = opts[:max_body]
receive do
{:gun_data, ^pid, ^stream, :fin, body} ->
check_body_size(acc, body, limit)
{:gun_data, ^pid, ^stream, :nofin, part} ->
- case check_body_size(acc, part, limit) do
- {:ok, acc} -> read_body(pid, stream, opts, acc)
- {:error, error} -> {:error, error}
+ with {:ok, acc} <- check_body_size(acc, part, limit) do
+ read_body(pid, stream, opts, acc)
end
+
+ {:DOWN, _, _, _, reason} ->
+ if opts[:close_conn], do: close(pid)
+ {:error, reason}
after
opts[:timeout] || @adapter_default_timeout ->
- {:error, :timeout}
+ {:error, :recv_body_timeout}
end
end
defp check_body_size(acc, part, nil), do: {:ok, acc <> part}
defp check_body_size(acc, part, limit) do
body = acc <> part
if limit - byte_size(body) >= 0 do
{:ok, body}
else
{:error, :body_too_large}
end
end
end
end
diff --git a/lib/tesla/adapter/shared.ex b/lib/tesla/adapter/shared.ex
index 379552f..1d75589 100644
--- a/lib/tesla/adapter/shared.ex
+++ b/lib/tesla/adapter/shared.ex
@@ -1,15 +1,24 @@
defmodule Tesla.Adapter.Shared do
@moduledoc false
def stream_to_fun(stream) do
reductor = fn item, _acc -> {:suspend, item} end
{_, _, fun} = Enumerable.reduce(stream, {:suspend, nil}, reductor)
fun
end
def next_chunk(fun), do: parse_chunk(fun.({:cont, nil}))
defp parse_chunk({:suspended, item, fun}), do: {:ok, item, fun}
defp parse_chunk(_), do: :eof
+
+ @spec prepare_path(String.t() | nil, String.t() | nil) :: String.t()
+ def prepare_path(nil, nil), do: "/"
+ def prepare_path(nil, query), do: "/?" <> query
+ def prepare_path(path, nil), do: path
+ def prepare_path(path, query), do: path <> "?" <> query
+
+ @spec format_method(atom()) :: String.t()
+ def format_method(method), do: to_string(method) |> String.upcase()
end
diff --git a/test/tesla/adapter/gun_test.exs b/test/tesla/adapter/gun_test.exs
index 9a005eb..228aa69 100644
--- a/test/tesla/adapter/gun_test.exs
+++ b/test/tesla/adapter/gun_test.exs
@@ -1,253 +1,341 @@
defmodule Tesla.Adapter.GunTest do
use ExUnit.Case
use Tesla.AdapterCase, adapter: Tesla.Adapter.Gun
use Tesla.AdapterCase.Basic
use Tesla.AdapterCase.Multipart
use Tesla.AdapterCase.StreamRequestBody
use Tesla.AdapterCase.SSL
alias Tesla.Adapter.Gun
test "fallback adapter timeout option" do
request = %Env{
method: :get,
url: "#{@http}/delay/2"
}
- assert {:error, :timeout} = call(request, timeout: 1_000)
+ assert {:error, :recv_response_timeout} = call(request, timeout: 1_000)
end
test "max_body option" do
request = %Env{
method: :get,
- url: "#{@http}/stream-bytes/100",
- query: [
- message: "Hello world!"
- ]
+ url: "#{@http}/stream-bytes/100"
}
assert {:error, :body_too_large} = call(request, max_body: 5)
end
test "query without path" do
request = %Env{
method: :get,
url: "#{@http}"
}
assert {:ok, %Env{} = response} = call(request)
assert response.status == 200
end
test "query without path with query" do
request = %Env{
method: :get,
url: "#{@http}",
query: [
param: "value"
]
}
assert {:ok, %Env{} = response} = call(request)
assert response.status == 200
end
test "response stream" do
request = %Env{
method: :get,
url: "#{@http}/stream-bytes/10"
}
assert {:ok, %Env{} = response} = call(request)
assert response.status == 200
assert byte_size(response.body) == 16
end
test "read response body in chunks" do
request = %Env{
method: :get,
url: "#{@http}/stream-bytes/10"
}
assert {:ok, %Env{} = response} = call(request, body_as: :chunks)
assert response.status == 200
%{pid: pid, stream: stream, opts: opts} = response.body
assert opts[:body_as] == :chunks
assert is_pid(pid)
assert is_reference(stream)
- assert read_body(pid, stream) |> byte_size() == 16
+ assert read_body(pid, stream, opts) |> byte_size() == 16
+ refute Process.alive?(pid)
+ end
+
+ test "read response body in chunks with closing connection by default opts" do
+ request = %Env{
+ method: :get,
+ url: "#{@http}/stream-bytes/10"
+ }
+
+ assert {:ok, %Env{} = response} = call(request, body_as: :chunks)
+ assert response.status == 200
+ %{pid: pid, stream: stream, opts: opts} = response.body
+ assert opts[:body_as] == :chunks
+ assert is_pid(pid)
+ assert is_reference(stream)
+
+ assert read_body(pid, stream, opts) |> byte_size() == 16
refute Process.alive?(pid)
end
- test "with body_as :plain reusinng connection" do
+ test "with body_as :plain reusing connection" do
uri = URI.parse(@http)
{:ok, conn} = :gun.open(to_charlist(uri.host), uri.port)
original = "#{uri.host}:#{uri.port}"
request = %Env{
method: :get,
url: "#{@http}/ip"
}
assert {:ok, %Env{} = response} =
call(request, conn: conn, close_conn: false, original: original)
assert response.status == 200
assert Process.alive?(conn)
assert {:ok, %Env{} = response} =
call(request, conn: conn, close_conn: false, original: original)
assert response.status == 200
assert Process.alive?(conn)
:ok = Gun.close(conn)
refute Process.alive?(conn)
end
test "read response body in chunks with reused connection and closing it" do
uri = URI.parse(@http)
{:ok, conn} = :gun.open(to_charlist(uri.host), uri.port)
original = "#{uri.host}:#{uri.port}"
request = %Env{
method: :get,
url: "#{@http}/stream-bytes/10"
}
assert {:ok, %Env{} = response} =
- call(request, body_as: :chunks, conn: conn, original: original)
+ call(request, body_as: :chunks, conn: conn, original: original, close_conn: false)
assert response.status == 200
%{pid: pid, stream: stream, opts: opts} = response.body
assert opts[:body_as] == :chunks
assert is_pid(pid)
assert is_reference(stream)
assert conn == pid
- assert read_body(pid, stream, "", false) |> byte_size() == 16
+ assert read_body(pid, stream, opts) |> byte_size() == 16
assert Process.alive?(pid)
# reusing connection
assert {:ok, %Env{} = response} =
- call(request, body_as: :chunks, conn: conn, original: original)
+ call(request, body_as: :chunks, conn: conn, original: original, close_conn: false)
assert response.status == 200
%{pid: pid, stream: stream, opts: opts} = response.body
assert opts[:body_as] == :chunks
assert is_pid(pid)
assert is_reference(stream)
assert conn == pid
- assert read_body(pid, stream, "", false) |> byte_size() == 16
+ assert read_body(pid, stream, opts) |> byte_size() == 16
assert Process.alive?(pid)
:ok = Gun.close(pid)
refute Process.alive?(pid)
end
test "don't reuse connection if original does not match" do
uri = URI.parse(@http)
{:ok, conn} = :gun.open(to_charlist(uri.host), uri.port)
request = %Env{
method: :get,
url: "#{@http}/stream-bytes/10"
}
assert {:ok, %Env{} = response} =
call(request, body_as: :chunks, conn: conn, original: "example.com:80")
assert response.status == 200
%{pid: pid, stream: stream, opts: opts} = response.body
assert opts[:body_as] == :chunks
assert is_pid(pid)
assert is_reference(stream)
- assert read_body(pid, stream, "", false) |> byte_size() == 16
- assert Process.alive?(pid)
+ assert read_body(pid, stream, opts) |> byte_size() == 16
+ refute Process.alive?(pid)
+ refute conn == pid
+ end
+
+ test "don't reuse connection and verify fun if original does not match in https request" do
+ uri = URI.parse(@https)
+
+ host = to_charlist(uri.host)
+
+ {:ok, conn} = :gun.open(host, uri.port)
+
+ request = %Env{
+ method: :get,
+ url: "#{@https}/stream-bytes/10"
+ }
+
+ assert {:ok, %Env{} = response} =
+ call(request,
+ body_as: :chunks,
+ conn: conn,
+ original: "example.com:443",
+ transport_opts: [
+ verify_fun: {&:ssl_verify_hostname.verify_fun/3, [check_hostname: 'example.com']}
+ ]
+ )
+
+ assert response.status == 200
+ %{pid: pid, stream: stream, opts: opts} = response.body
+
+ assert is_pid(pid)
+ assert is_reference(stream)
+
+ assert read_body(pid, stream, opts) |> byte_size() == 16
+
+ refute Process.alive?(pid)
+ assert opts[:old_conn] == conn
refute conn == pid
end
+ test "certificates_verification" do
+ request = %Env{
+ method: :get,
+ url: "#{@https}"
+ }
+
+ assert {:ok, %Env{} = response} =
+ call(request,
+ certificates_verification: true,
+ transport_opts: [
+ verify_fun:
+ {fn
+ _cert, _reason, state ->
+ {:valid, state}
+ end, nil}
+ ]
+ )
+ end
+
test "read response body in stream" do
request = %Env{
method: :get,
url: "#{@http}/stream-bytes/10"
}
assert {:ok, %Env{} = response} = call(request, body_as: :stream)
assert response.status == 200
assert is_function(response.body)
- assert Enum.to_list(response.body) |> List.to_string() |> byte_size() == 16
+ assert reduce_body(response.body) |> byte_size() == 16
end
test "read response body in stream with opened connection without closing connection" do
uri = URI.parse(@http)
{:ok, conn} = :gun.open(to_charlist(uri.host), uri.port)
original = "#{uri.host}:#{uri.port}"
request = %Env{
method: :get,
url: "#{@http}/stream-bytes/10"
}
assert {:ok, %Env{} = response} =
call(request, body_as: :stream, conn: conn, close_conn: false, original: original)
assert response.status == 200
assert is_function(response.body)
- assert Enum.to_list(response.body) |> List.to_string() |> byte_size() == 16
+ assert reduce_body(response.body) |> byte_size() == 16
assert Process.alive?(conn)
:ok = Gun.close(conn)
refute Process.alive?(conn)
end
test "read response body in stream with opened connection with closing connection" do
uri = URI.parse(@http)
{:ok, conn} = :gun.open(to_charlist(uri.host), uri.port)
original = "#{uri.host}:#{uri.port}"
request = %Env{
method: :get,
url: "#{@http}/stream-bytes/10"
}
assert {:ok, %Env{} = response} =
call(request, body_as: :stream, conn: conn, original: original)
assert response.status == 200
assert is_function(response.body)
- assert Enum.to_list(response.body) |> List.to_string() |> byte_size() == 16
+ assert reduce_body(response.body) |> byte_size() == 16
refute Process.alive?(conn)
end
test "error response" do
request = %Env{
method: :get,
url: "#{@http}/status/500"
}
assert {:ok, %Env{} = response} = call(request, timeout: 1_000)
assert response.status == 500
end
- defp read_body(pid, stream, acc \\ "", close_conn \\ true) do
- case Gun.read_chunk(pid, stream, timeout: 1_000) do
- {:fin, body} ->
- if close_conn do
- :ok = Gun.close(pid)
- end
+ test "error on socks proxy" do
+ request = %Env{
+ method: :get,
+ url: "#{@http}/status/500"
+ }
+
+ assert {:error, "socks protocol is not supported"} ==
+ call(request, proxy: {:socks5, 'localhost', 1234})
+ end
+
+ test "receive gun_up message when receive is false" do
+ request = %Env{
+ method: :get,
+ url: "#{@http}"
+ }
+
+ assert {:ok, %Env{} = response} = call(request, receive: false)
+ assert response.status == 200
+ assert_receive {:gun_up, pid, :http}
+ assert is_pid(pid)
+ end
+ defp read_body(pid, stream, opts, acc \\ "") do
+ case Gun.read_chunk(pid, stream, opts) do
+ {:fin, body} ->
acc <> body
{:nofin, part} ->
- read_body(pid, stream, acc <> part, close_conn)
+ read_body(pid, stream, opts, acc <> part)
end
end
+
+ defp reduce_body(body), do: Enum.reduce(body, "", fn part, acc -> acc <> part end)
end

File Metadata

Mime Type
text/x-diff
Expires
Tue, Nov 26, 12:20 AM (1 d, 9 h)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
40125
Default Alt Text
(46 KB)

Event Timeline