Page MenuHomePhorge
Files F113915

No OneTemporary
Actions

Size
32 KB
Referenced Files
None
Subscribers
None

View Options

diff --git a/README.md b/README.md
index 5e94a60..2195ed7 100644
--- a/README.md
+++ b/README.md
@@ -1,194 +1,195 @@
# Majic
**Majic** provides a robust integration of [libmagic](http://man7.org/linux/man-pages/man3/libmagic.3.html) for Elixir.
With this library, you can start an one-off process to run a single check, or run the process as a daemon if you expect to run
many checks.
It is a friendly fork of [gen_magic](https://github.com/evadne/gen_magic) featuring a (arguably) more robust C-code
using erl_interface, built in pooling, unified/clean API, and an optional Plug.
This package is regulary tested on multiple platforms (Debian, macOS, Fedora, Alpine, FreeBSD) to ensure it'll work fine
in any environment.
-Majic depends on [`file`](https://github.com/file/file) mirror repository to provide an up-to-date magic database. The database is compiled when Majic is compiled.
+Majic depends on [`file`](https://github.com/file/file) mirror repository to provide an up-to-date magic database. The database is compiled when Majic is compiled. We frequently update
+the database provided, and as well have additional temporary patches in `src/magic_patches/`.
## Installation
The package can be installed by adding `majic` to your list of dependencies in `mix.exs`:
```elixir
def deps do
[
{:majic, "~> 1.0"}
]
end
```
You must also have [libmagic](http://man7.org/linux/man-pages/man3/libmagic.3.html) installed locally with headers, alongside common compilation tools (i.e. build-essential). These can be acquired by apt-get, yum, brew, etc.
Compilation of the underlying C program is automatic and handled by [elixir_make](https://github.com/elixir-lang/elixir_make).
## Usage
Depending on the use case, you may utilise a single (one-off) Majic process without reusing it as a daemon, or utilise a connection pool (such as Poolboy) in your application to run multiple persistent Majic processes.
To use Majic directly, you can use `Majic.Once.perform/1`:
```elixir
iex(1)> Majic.perform(".", once: true)
{:ok,
%Majic.Result{
content: "directory",
encoding: "binary",
mime_type: "inode/directory"
}}
```
To use the Majic server as a daemon, you can start it first, keep a reference, then feed messages to it as you require:
```elixir
{:ok, pid} = Majic.Server.start_link([])
{:ok, result} = Majic.perform(path, server: pid)
```
See `Majic.Server.start_link/1` and `t:Majic.Server.option/0` for more information on startup parameters.
See `Majic.Result` for details on the result provided.
## Configuration
When using `Majic.Server.start_link/1` to start a persistent server, or `Majic.Helpers.perform_once/2` to run an ad-hoc request, you can override specific options to suit your use case.
| Name | Default | Description |
| - | - | - |
| `:startup_timeout` | 1000 | Number of milliseconds to wait for client startup |
| `:process_timeout` | 30000 | Number of milliseconds to process each request |
| `:recycle_threshold` | 10 | Number of cycles before the C process is replaced |
| `:database_patterns` | `[:default]` | Databases to load |
See `t:Majic.Server.option/0` for details.
__Note__ `:recycle_thresold` is only useful if you are using a libmagic `<5.29`, where it was susceptible to memleaks
([details](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=840754)]). In future versions of `majic` this option could
be ignored.
### Reloading / Altering databases
If you want `majic` to reload its database(s), run `Majic.Server.reload(ref)`.
If you want to add or remove databases to a running server, you would have to run `Majic.Server.reload(ref, databases)`
where databases being the same argument as `database_patterns` on start. `Majic` does not support adding/removing
databases at runtime without a port reload.
### Use Cases
#### Ad-Hoc Requests
For ad-hoc requests, you can use the helper method `Majic.Once.perform_once/2`:
```elixir
iex(1)> Majic.perform(Path.join(File.cwd!(), "Makefile"), once: true)
{:ok,
%Majic.Result{
content: "makefile script, ASCII text",
encoding: "us-ascii",
mime_type: "text/x-makefile"
}}
```
#### Supervised Requests
The Server should be run under a supervisor which provides resiliency.
Here we run it under a supervisor in an application:
```elixir
children =
[
# ...
{Majic.Server, [name: YourApp.Majic]}
]
opts = [strategy: :one_for_one, name: YourApp.Supervisor]
Supervisor.start_link(children, opts)
```
Now we can ask it to inspect a file:
```elixir
iex(2)> Majic.perform(Path.expand("~/.bash_history"), server: YourApp.Majic)
{:ok, %Majic.Result{mime_type: "text/plain", encoding: "us-ascii", content: "ASCII text"}}
```
Note that in this case we have opted to use a named process.
#### Pool
For concurrency *and* resiliency, you may start the `Majic.Pool`. By default, it will start a `Majic.Server`
worker per online scheduler:
You can add a pool in your application supervisor by adding it as a child:
```elixir
children =
[
# ...
{Majic.Pool, [name: YourApp.MajicPool, pool_size: 2]}
]
opts = [strategy: :one_for_one, name: YourApp.Supervisor]
Supervisor.start_link(children, opts)
```
And then you can use it with `Majic.perform/2` with `pool: YourApp.MajicPool` option:
```elixir
iex(1)> Majic.perform(Path.expand("~/.bash_history"), pool: YourApp.MajicPool)
{:ok, %Majic.Result{mime_type: "text/plain", encoding: "us-ascii", content: "ASCII text"}}
```
#### Fixing extensions
You may also want to fix the user-provided filename according to its detected MIME type. To do this, you can use `Majic.Extension.fix/3`:
```elixir
iex(1)> {:ok, result} = Majic.perform("cat.jpeg", once: true)
{:ok, %Majic.Result{mime_type: "image/webp", ...}}
iex(1)> Majic.Extension.fix("cat.jpeg", result)
"cat.webp"
```
#### Use with Plug.Upload
If you use Plug or Phoenix, you may want to automatically verify the content type of every `Plug.Upload`. The
`Majic.Plug` is there for this.
Enable it by using `plug Majic.Plug, pool: YourApp.MajicPool` in your pipeline or controller. Then, every `Plug.Upload`
in `conn.params` and `conn.body_params` is now verified. The filename is also altered with an extension matching its
content-type, using `Majic.Extension`.
## Notes
### Soak Test
Run an endless cycle to prove that the program is resilient:
```bash
find /usr/share/ -name *png | xargs mix run test/soak.exs
find . -name *ex | xargs mix run test/soak.exs
```
## Acknowledgements
During design and prototype development of this library, the Author has drawn inspiration from the following individuals, and therefore
thanks all contributors for their generosity:
- [Evadne Wu](https://github.com/evadne)
- Original [gen_magic](https://github.com/evadne/gen_magic) author.
- [James Every](https://github.com/devstopfix)
- Enhanced Elixir Wrapper (based on GenServer)
- Initial Hex packaging (v.0.22)
- Soak Testing
- Matthias and Ced for helping the author with C oddities
- [Hecate](https://github.com/Kleidukos) for laughing at aforementionned oddities
- majic for giving inspiration for the lib name (magic, majic, get it? hahaha..)
diff --git a/lib/majic.ex b/lib/majic.ex
index 5c2651e..49ba26d 100644
--- a/lib/majic.ex
+++ b/lib/majic.ex
@@ -1,79 +1,89 @@
defmodule Majic do
alias Majic.{Once, Pool, Result, Server}
@moduledoc """
Robust libmagic integration for Elixir.
"""
@doc """
Perform on `path`.
An option of `server: ServerName`, `pool: PoolName` or `once: true` must be passed.
"""
@type target :: Path.t() | {:bytes, binary()}
@type result :: {:ok, Result.t()} | {:error, term() | String.t()}
@type name :: {:pool, atom()} | {:server, Server.t()} | {:once, true}
@type option :: name | Server.start_option() | Pool.option()
@spec perform(target(), [option()]) :: result()
def perform(path, opts) do
mod =
cond do
Keyword.has_key?(opts, :pool) -> {Pool, Keyword.get(opts, :pool)}
Keyword.has_key?(opts, :server) -> {Server, Keyword.get(opts, :server)}
Keyword.has_key?(opts, :once) -> {Once, nil}
true -> nil
end
opts =
opts
|> Keyword.drop([:pool, :server, :once])
if mod do
do_perform(mod, path, opts)
else
{:error, :no_method}
end
end
@doc "Compiles a `magic` file or a `Magdir` directory to a magic-compiled database (`.mgc`)"
def compile(path, timeout \\ 5_000) do
- port = Port.open(Majic.Config.get_port_name(), [:use_stdio, :binary, :exit_status, {:packet, 2}, {:args, []}])
+ port =
+ Port.open(Majic.Config.get_port_name(), [
+ :use_stdio,
+ :binary,
+ :exit_status,
+ {:packet, 2},
+ {:args, []}
+ ])
+
compile(port, path, timeout)
end
defp compile(port, path, timeout) do
receive do
{^port, {:data, data}} ->
case :erlang.binary_to_term(data) do
:ready ->
send(port, {self(), {:command, :erlang.term_to_binary({:compile_database, path})}})
+
receive do
{^port, {:data, data}} ->
:erlang.binary_to_term(data)
after
timeout ->
{:error, :timeout}
end
+
result ->
result
end
- after
- timeout ->
- {:error, :timeout}
+ after
+ timeout ->
+ {:error, :timeout}
+ end
end
- end
defp do_perform({Server = mod, name}, path, opts) do
timeout = Keyword.get(opts, :timeout, Majic.Config.default_process_timeout())
mod.perform(name, path, timeout)
end
defp do_perform({Once = mod, _}, path, opts) do
mod.perform(path, opts)
end
defp do_perform({Pool = mod, name}, path, opts) do
mod.perform(name, path, opts)
end
end
diff --git a/lib/majic/server.ex b/lib/majic/server.ex
index 4a8c29a..e0e7920 100644
--- a/lib/majic/server.ex
+++ b/lib/majic/server.ex
@@ -1,449 +1,450 @@
defmodule Majic.Server do
@moduledoc """
Provides access to the underlying libmagic client, which performs file introspection.
The Server needs to be supervised, since it will terminate if it receives any unexpected error.
"""
@behaviour :gen_statem
alias Majic.Result
alias Majic.Server.Data
alias Majic.Server.Status
import Kernel, except: [send: 2]
@database_patterns [:default]
@process_timeout Majic.Config.default_process_timeout()
@typedoc """
Represents the reference to the underlying server, as returned by `:gen_statem`.
"""
@type t :: :gen_statem.server_ref()
@typedoc """
Represents values accepted as startup options, which can be passed to `start_link/1`.
- `:name`: If present, this will be the registered name for the underlying process.
Note that `:gen_statem` requires `{:local, name}`, but given widespread GenServer convention,
atoms are accepted and will be converted to `{:local, name}`.
- `:startup_timeout`: Specifies how long the Server waits for the C program to initialise.
However, if the underlying C program exits, then the process exits immediately.
Can be set to `:infinity`.
- `:process_timeout`: Specifies how long the Server waits for each request to complete.
Can be set to `:infinity`.
Please note that, if you have chosen a custom timeout value, you should also pass it when
using `Majic.Server.perform/3`.
- `:recycle_threshold`: Specifies the number of requests processed before the underlying C
program is recycled.
Can be set to `:infinity` if you do not wish for the program to be recycled.
- `:database_patterns`: Specifies what magic databases to load; you can specify a list of files, or of
Path Patterns (see `Path.wildcard/2`) or `:default` to instruct the C program to load the
appropriate databases.
For example, if you have had to add custom magics, then you can set this value to:
[:default, "path/to/my/magic"]
"""
@type start_option ::
{:name, atom() | :gen_statem.server_name()}
| {:startup_timeout, timeout()}
| {:process_timeout, timeout()}
| {:recycle_threshold, non_neg_integer() | :infinity}
| {:database_patterns, nonempty_list(:default | :system | Path.t())}
@typedoc """
Current state of the Server:
- `:pending`: This is the initial state; the Server will attempt to start the underlying Port
and the libmagic client, then automatically transition to either Available or Crashed.
- `:available`: This is the default state. In this state the Server is able to accept requests
and they will be replied in the same order.
- `:processing`: This is the state the Server will be in if it is processing requests. In this
state, further requests can still be lodged and they will be processed when the Server is
available again.
For proper concurrency, use a process pool like Poolboy, Sbroker, etc.
- `:recycling`: This is the state the Server will be in, if its underlying C program needs to be
recycled. This state is triggered whenever the cycle count reaches the defined value as per
`:recycle_threshold`.
In this state, the Server is able to accept requests, but they will not be processed until the
underlying C server program has been started again.
"""
@type state :: :starting | :processing | :available | :recycling
@spec child_spec([start_option()]) :: Supervisor.child_spec()
@spec start_link([start_option()]) :: :gen_statem.start_ret()
@spec perform(t(), Majic.target(), timeout()) :: Majic.result()
@spec status(t(), timeout()) :: {:ok, Status.t()} | {:error, term()}
@spec stop(t(), term(), timeout()) :: :ok
@doc """
Returns the default Child Specification for this Server for use in Supervisors.
You can override this with `Supervisor.child_spec/2` as required.
"""
def child_spec(options) do
%{
id: __MODULE__,
start: {__MODULE__, :start_link, [options]},
type: :worker,
restart: :permanent,
shutdown: 500
}
end
@doc """
Starts a new Server.
See `t:option/0` for further details.
"""
def start_link(options) do
{name, options} = Keyword.pop(options, :name)
case name do
nil -> :gen_statem.start_link(__MODULE__, options, [])
name when is_atom(name) -> :gen_statem.start_link({:local, name}, __MODULE__, options, [])
{:global, _} -> :gen_statem.start_link(name, __MODULE__, options, [])
{:via, _, _} -> :gen_statem.start_link(name, __MODULE__, options, [])
{:local, _} -> :gen_statem.start_link(name, __MODULE__, options, [])
end
end
@doc """
Determines the type of the file provided.
"""
def perform(server_ref, path, timeout \\ @process_timeout) do
case :gen_statem.call(server_ref, {:perform, path}, timeout) do
{:ok, %Result{} = result} -> {:ok, result}
{:error, reason} -> {:error, reason}
end
end
@doc """
Reloads a Server with a new set of databases.
"""
def reload(server_ref, database_patterns \\ nil, timeout \\ @process_timeout) do
:gen_statem.call(server_ref, {:reload, database_patterns}, timeout)
end
@doc """
Same as `reload/2,3` but with a full restart of the underlying C port.
"""
def recycle(server_ref, database_patterns \\ nil, timeout \\ @process_timeout) do
:gen_statem.call(server_ref, {:recycle, database_patterns}, timeout)
end
@doc """
Returns status of the Server.
"""
def status(server_ref, timeout \\ @process_timeout) do
:gen_statem.call(server_ref, :status, timeout)
end
@doc """
Stops the Server with reason `:normal` and timeout `:infinity`.
"""
def stop(server_ref) do
:gen_statem.stop(server_ref)
end
@doc """
Stops the Server with the specified reason and timeout.
"""
def stop(server_ref, reason, timeout) do
:gen_statem.stop(server_ref, reason, timeout)
end
@impl :gen_statem
def init(options) do
import Majic.Config
data = %Data{
port_name: get_port_name(),
database_patterns: Keyword.get(options, :database_patterns),
port_options: get_port_options(options),
startup_timeout: get_startup_timeout(options),
process_timeout: get_process_timeout(options),
recycle_threshold: get_recycle_threshold(options)
}
{:ok, :starting, data}
end
@impl :gen_statem
def callback_mode do
[:state_functions, :state_enter]
end
@doc false
def starting(:enter, _, %{port: nil} = data) do
port = Port.open(data.port_name, data.port_options)
{:keep_state, %{data | port: port}, data.startup_timeout}
end
@doc false
def starting(:enter, _, data) do
{:keep_state_and_data, data.startup_timeout}
end
@doc false
def starting({:call, from}, :status, data) do
handle_status_call(from, :starting, data)
end
@doc false
def starting({:call, _from}, {:perform, _path}, _data) do
{:keep_state_and_data, :postpone}
end
@doc false
def starting(:info, {port, {:data, ready}}, %{port: port} = data) do
case :erlang.binary_to_term(ready) do
:ready -> {:next_state, :loading, data}
end
end
@doc false
def starting(:info, {port, {:exit_status, code}}, %{port: port} = data) do
error =
case code do
1 -> :bad_db
2 -> :ei_error
3 -> :ei_bad_term
4 -> :magic_error
code -> {:unexpected_error, code}
end
{:stop, {:error, error}, data}
end
@doc false
def loading(:enter, _old_state, data) do
databases =
Enum.flat_map(List.wrap(data.database_patterns || @database_patterns), fn
:default -> [:default]
:system -> [:system]
pattern -> Path.wildcard(pattern)
end)
if databases == [] do
{:stop, {:error, :no_databases_to_load}, data}
else
{:keep_state, {databases, data}, {:state_timeout, 0, :load}}
end
end
@doc false
def loading(:state_timeout, :load_timeout, {[database | _], data}) do
{:stop, {:error, {:database_loading_timeout, database}}, data}
end
@doc false
def loading(:state_timeout, :load, {[], data}) do
{:next_state, :available, data}
end
@doc false
def loading(:state_timeout, :load, {[database | _databases], data} = state) do
priv_dir = to_string(:code.priv_dir(:majic))
+
command =
case database do
:default -> {:add_database, Path.join(priv_dir, "/magic.mgc")}
:system -> {:add_database, :default}
path when is_binary(path) -> {:add_database, path}
end
send(data.port, command)
{:keep_state, state, {:state_timeout, data.startup_timeout, :load_timeout}}
end
@doc false
def loading(:info, {port, {:data, response}}, {[database | databases], %{port: port} = data}) do
case :erlang.binary_to_term(response) do
{:ok, :loaded} ->
{:keep_state, {databases, data}, {:state_timeout, 0, :load}}
{:error, :not_loaded} ->
{:stop, {:error, {:database_load_failed, database}}, data}
end
end
@doc false
def loading({:call, from}, :status, {_, data}) do
handle_status_call(from, :loading, data)
end
@doc false
def loading({:call, _from}, {:perform, _path}, _data) do
{:keep_state_and_data, :postpone}
end
@doc false
def available(:enter, _old_state, %{request: {:reload, from, _}}) do
response = {:reply, from, :ok}
{:keep_state_and_data, response}
end
@doc false
def available(:enter, _old_state, %{request: nil}) do
:keep_state_and_data
end
@doc false
def available({:call, from}, {:perform, path}, data) do
data = %{data | cycles: data.cycles + 1, request: {path, from, :erlang.now()}}
arg =
case path do
path when is_binary(path) -> {:file, path}
# Truncate to 50 bytes
{:bytes, <<bytes::binary-size(50), _::binary>>} -> {:bytes, bytes}
{:bytes, bytes} -> {:bytes, bytes}
end
send(data.port, arg)
{:next_state, :processing, data}
end
@doc false
def available({:call, from}, {:reload, databases}, data) do
send(data.port, {:reload, :reload})
{:next_state, :starting,
%{
data
| database_patterns: databases || data.database_patterns,
request: {:reload, from, :reload}
}}
end
@doc false
def available({:call, from}, {:recycle, databases}, data) do
{:next_state, :recycling,
%{
data
| database_patterns: databases || data.database_patterns,
request: {:reload, from, :recycle}
}}
end
@doc false
def available({:call, from}, :status, data) do
handle_status_call(from, :available, data)
end
@doc false
def processing(:enter, _old_state, %{request: {_path, _from, _time}} = data) do
{:keep_state_and_data, data.process_timeout}
end
@doc false
def processing({:call, _from}, {:perform, _path}, _data) do
{:keep_state_and_data, :postpone}
end
@doc false
def processing({:call, from}, :status, data) do
handle_status_call(from, :processing, data)
end
@doc false
def processing(:state_timeout, _, %{request: {_, from, _}} = data) do
response = {:reply, from, {:error, :timeout}}
{:next_state, :recycling, %{data | request: nil}, [response, :hibernate]}
end
@doc false
def processing(:info, {port, {:data, response}}, %{port: port, request: {_, from, _}} = data) do
response = {:reply, from, handle_response(response)}
next_state = (data.cycles >= data.recycle_threshold && :recycling) || :available
{:next_state, next_state, %{data | request: nil}, [response, :hibernate]}
end
@doc false
def recycling(:enter, _, %{port: port} = data) when is_port(port) do
send(data.port, {:stop, :recycle})
{:keep_state_and_data, {:state_timeout, data.startup_timeout, :stop}}
end
@doc false
def recycling({:call, _from}, {:perform, _path}, _data) do
{:keep_state_and_data, :postpone}
end
@doc false
def recycling({:call, from}, :status, data) do
handle_status_call(from, :recycling, data)
end
@doc false
# In case of timeout, force close.
def recycling(:state_timeout, :stop, data) do
Kernel.send(data.port, {self(), :close})
{:keep_state_and_data, {:state_timeout, data.startup_timeout, :close}}
end
@doc false
def recycling(:state_timeout, :close, data) do
{:stop, {:error, :port_close_failed}, data}
end
@doc false
def recycling(:info, {port, :closed}, %{port: port} = data) do
{:next_state, :starting, %{data | port: nil, cycles: 0}}
end
@doc false
def recycling(:info, {port, {:exit_status, _}}, %{port: port} = data) do
{:next_state, :starting, %{data | port: nil, cycles: 0}}
end
@doc false
@impl :gen_statem
def terminate(_, _, %{port: port}) do
Kernel.send(port, {self(), :close})
end
@doc false
def terminate(_, _, _) do
:ok
end
defp send(port, command) do
Kernel.send(port, {self(), {:command, :erlang.term_to_binary(command)}})
end
@errnos %{
2 => :enoent,
13 => :eaccess,
20 => :enotdir,
12 => :enomem,
24 => :emfile,
36 => :enametoolong
}
@errno Map.keys(@errnos)
defp handle_response(data) do
case :erlang.binary_to_term(data) do
{:ok, {mime_type, encoding, content}} -> {:ok, Result.build(mime_type, encoding, content)}
{:error, {errno, _}} when errno in @errno -> {:error, @errnos[errno]}
{:error, {errno, string}} -> {:error, "#{errno}: #{string}"}
{:error, _} = error -> error
end
end
defp handle_status_call(from, state, data) do
response = {:ok, %__MODULE__.Status{state: state, cycles: data.cycles}}
{:keep_state_and_data, {:reply, from, response}}
end
end
diff --git a/lib/mix/tasks/compile/majic.ex b/lib/mix/tasks/compile/majic.ex
index 75a930d..4db8312 100644
--- a/lib/mix/tasks/compile/majic.ex
+++ b/lib/mix/tasks/compile/majic.ex
@@ -1,108 +1,128 @@
defmodule Mix.Tasks.Compile.Majic do
use Mix.Task.Compiler
- @repo_path Path.join(Mix.Project.deps_path, "libfile")
+ @repo_path Path.join(Mix.Project.deps_path(), "libfile")
@magic_path Path.join(@repo_path, "/magic")
@magdir Path.join(@magic_path, "/Magdir")
- @build_path Path.join(Mix.Project.build_path, "/majic")
+ @build_path Path.join(Mix.Project.build_path(), "/majic")
@build_magdir Path.join(@build_path, "/magic")
- @manifest Path.join(@build_path, "/majic.manifest")
+ @manifest Path.join(Mix.Project.manifest_path(), "/compile.majic")
@patch_path "src/magic_patches"
@built_path Path.join(to_string(:code.priv_dir(:majic)), "/magic.mgc")
@shortdoc "Updates and compiles majic's embedded magic database."
- @doc """
- Uses `libfile` dependency Magdir, applies patches from `#{@patch_path}`, and builds to `#{@built_path}`.
+ @moduledoc """
+ Uses `libfile` dependency Magdir, applies patches from `#{@patch_path}`, and builds to `#{
+ @built_path
+ }`.
"""
defmodule Manifest do
+ @moduledoc false
defstruct [:hash, {:patches, []}]
end
@impl Mix.Task.Compiler
- def clean() do
+ def clean do
File.rm!(@built_path)
File.rm_rf!(@build_path)
end
@impl Mix.Task.Compiler
- def manifests() do
+ def manifests do
[@manifest]
end
@impl Mix.Task.Compiler
def run(_) do
{:ok, manifest} = read_manifest()
{:ok, sha} = get_dep_revision()
patches = list_patches()
- File.mkdir_p!(Path.join(@build_path, "/magic"))
+ File.mkdir_p!(@build_magdir)
+
if sha == nil || sha != manifest.hash || patches != manifest.patches do
:ok = assemble_magdir()
{:ok, patches, _err} = apply_patches()
{:ok, _} = Majic.compile(@build_magdir)
- File.rm_rf!(@build_magdir)
File.cp!("magic.mgc", @built_path)
File.rm!("magic.mgc")
manifest = %Manifest{hash: sha, patches: Enum.sort(patches)}
File.write!(@manifest, :erlang.term_to_binary(manifest))
Mix.shell().info("Generated magic database")
else
Mix.shell().info("Magic database up-to-date")
end
+
:ok
end
- defp read_manifest() do
+ defp read_manifest do
with {:ok, binary} <- File.read(@manifest),
- {:term, %Manifest{} = manifest} <- {:term, :erlang.binary_to_term(binary)}
- do
+ {:term, %Manifest{} = manifest} <- {:term, :erlang.binary_to_term(binary)} do
{:ok, manifest}
else
{:term, _} -> {:ok, %Manifest{}}
{:error, :enoent} -> {:ok, %Manifest{}}
error -> error
end
end
- defp get_dep_revision() do
+ defp get_dep_revision do
git_dir = Path.join(@repo_path, "/.git")
+
case System.cmd("git", ["--git-dir=#{git_dir}", "rev-parse", "HEAD"]) do
{rev, 0} -> {:ok, String.trim(rev)}
_ -> {:ok, nil}
end
end
- defp list_patches() do
+ defp list_patches do
@patch_path
|> File.ls!()
+ |> Enum.filter(fn file ->
+ Path.extname(file) == ".patch"
+ end)
|> Enum.sort()
end
- defp apply_patches() do
- patched = list_patches()
- |> Enum.map(fn(patch) ->
- path = Path.expand(Path.join(@patch_path, patch))
- case System.cmd("git", ["apply", path], cd: @build_path) do
- {_, 0} ->
- Mix.shell().info("Patched magic database: #{patch}")
- {patch, :ok}
- {error, code} ->
- Mix.shell().error("Failed to apply patch #{patch} (#{code})")
- Mix.shell().error(error)
- {patch, {:error, error}}
- end
- end)
- ok = Enum.filter(patched, fn({_, result}) -> result == :ok end) |> Enum.map(fn({name, _}) -> name end)
- err = Enum.filter(patched, fn({_, result}) -> result != :ok end) |> Enum.map(fn({name, {:error, error}}) -> {name, error} end)
+ defp apply_patches do
+ patched =
+ list_patches()
+ |> Enum.map(fn patch ->
+ path = Path.expand(Path.join(@patch_path, patch))
+
+ # We rewrite paths in patch because we're not applying the patch from repository root, but in a temporary
+ # build folder that is just the Magdir.
+ case System.cmd("git", ["apply", "-p3", "--directory=magic/", path], cd: @build_path) do
+ {_, 0} ->
+ Mix.shell().info("Patched magic database: #{patch}")
+ {patch, :ok}
+
+ {error, code} ->
+ Mix.shell().error("Failed to apply patch #{patch} (#{code})")
+ Mix.shell().error(error)
+ {patch, {:error, error}}
+ end
+ end)
+
+ ok =
+ Enum.filter(patched, fn {_, result} -> result == :ok end)
+ |> Enum.map(fn {name, _} -> name end)
+
+ err =
+ Enum.filter(patched, fn {_, result} -> result != :ok end)
+ |> Enum.map(fn {name, {:error, error}} -> {name, error} end)
+
{:ok, ok, err}
end
- defp assemble_magdir() do
+ defp assemble_magdir do
File.rm_rf!(@build_magdir)
File.mkdir!(@build_magdir)
+
File.ls!(@magdir)
- |> Enum.each(fn(file) ->
- File.cp!(Path.join(@magdir, file), Path.join(@build_magdir, file))
+ |> Enum.each(fn file ->
+ File.cp!(Path.join(@magdir, file), Path.join(@build_magdir, file))
end)
+
:ok
end
-
end
diff --git a/mix.exs b/mix.exs
index a1ead9b..48a6386 100644
--- a/mix.exs
+++ b/mix.exs
@@ -1,85 +1,86 @@
defmodule Majic.MixProject do
use Mix.Project
if :erlang.system_info(:otp_release) < '21' do
raise "Majic requires Erlang/OTP 21 or newer"
end
def project do
[
app: :majic,
version: "1.0.0",
elixir: "~> 1.7",
elixirc_paths: elixirc_paths(Mix.env()),
elixirc_options: [warnings_as_errors: warnings_as_errors(Mix.env())],
start_permanent: Mix.env() == :prod,
compilers: [:elixir_make] ++ Mix.compilers() ++ [:majic],
make_env: make_env(),
package: package(),
deps: deps(),
dialyzer: dialyzer(),
name: "Majic",
description: "File introspection with libmagic",
source_url: "https://github.com/hrefhref/majic",
docs: docs()
]
end
def application do
[extra_applications: [:logger]]
end
defp elixirc_paths(:test), do: ["lib", "test/support"]
defp elixirc_paths(_), do: ["lib"]
defp dialyzer do
[
plt_add_apps: [:mix, :iex, :ex_unit, :plug, :mime],
flags: ~w(error_handling no_opaque race_conditions underspecs unmatched_returns)a,
ignore_warnings: "dialyzer-ignore-warnings.exs",
list_unused_filters: true
]
end
defp deps do
[
{:nimble_pool, "~> 0.1"},
{:mime, "~> 1.0"},
{:plug, "~> 1.0", optional: true},
{:credo, "~> 1.4", only: [:dev, :test], runtime: false},
{:dialyxir, "~> 1.0.0-rc.6", only: :dev, runtime: false},
{:ex_doc, ">= 0.0.0", only: :dev, runtime: false},
{:elixir_make, "~> 0.6.1", runtime: false},
- {:libfile, github: "file/file", branch: "master", app: false, compile: false, sparse: "magic/"}
+ {:libfile,
+ github: "file/file", branch: "master", app: false, compile: false, sparse: "magic/"}
]
end
defp package do
[
files: ~w(lib/* src/* Makefile),
licenses: ["Apache 2.0"],
links: %{"GitHub" => "https://github.com/hrefhref/majic"},
source_url: "https://github.com/hrefhref/majic"
]
end
defp docs do
[
main: "readme",
extras: ["README.md", "CHANGELOG.md"]
]
end
defp warnings_as_errors(:dev), do: false
defp warnings_as_errors(_), do: true
defp make_env() do
- otp = :erlang.system_info(:otp_release)
- |> to_string()
- |> String.to_integer()
+ otp =
+ :erlang.system_info(:otp_release)
+ |> to_string()
+ |> String.to_integer()
ei_incomplete = if(otp < 21.3, do: "YES", else: "NO")
%{"EI_INCOMPLETE" => ei_incomplete}
end
-
end
diff --git a/src/magic_patches/README.md b/src/magic_patches/README.md
new file mode 100644
index 0000000..8dab476
--- /dev/null
+++ b/src/magic_patches/README.md
@@ -0,0 +1,4 @@
+# majic Magdir patches
+
+* Only patches working entierly on `/magic/Magdir/` can be accepted.
+* Only patches posted on the [`file` mailing list](https://mailman.astron.com/mailman/listinfo/file) will be accepted.

File Metadata

Mime Type
text/x-diff
Expires
Mon, Nov 25, 4:24 PM (1 d, 7 h)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
39961
Default Alt Text
(32 KB)

Event Timeline