Page MenuHomePhorge

No OneTemporary

Authored By
Unknown
Size
14 KB
Referenced Files
None
Subscribers
None
diff --git a/Dockerfile b/Dockerfile
index 87c698f93..68c8a2f80 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -13,7 +13,9 @@ COPY . .
ENV MIX_ENV=prod
ENV VIX_COMPILATION_MODE=PLATFORM_PROVIDED_LIBVIPS
-RUN apk add git gcc g++ musl-dev make cmake file-dev vips-dev &&\
+# vips-heif: libvips' HEIF/AVIF module, also at build time (Vix makes its
+# functions from what libvips offers then; AVIF previews need heifsave)
+RUN apk add git gcc g++ musl-dev make cmake file-dev vips-dev vips-heif &&\
echo "import Config" > config/prod.secret.exs &&\
mix local.hex --force &&\
mix local.rebar --force &&\
@@ -42,7 +44,7 @@ ARG HOME=/opt/pleroma
ARG DATA=/var/lib/pleroma
RUN apk update &&\
- apk add exiftool ffmpeg vips libmagic ncurses postgresql-client &&\
+ apk add exiftool ffmpeg vips vips-heif libmagic ncurses postgresql-client &&\
adduser --system --shell /bin/false --home ${HOME} pleroma &&\
mkdir -p ${DATA}/uploads &&\
mkdir -p ${DATA}/static &&\
diff --git a/changelog.d/media-preview-image-sizes.add b/changelog.d/media-preview-image-sizes.add
index 9da276833..fe04ee504 100644
--- a/changelog.d/media-preview-image-sizes.add
+++ b/changelog.d/media-preview-image-sizes.add
@@ -1 +1 @@
-Media preview proxy: image previews in configured sizes besides the thumbnail (`image_sizes`), listed on image attachments as `pleroma.image_sizes`.
+Media preview proxy: image previews in configured sizes besides the thumbnail (`image_sizes`), listed on image attachments as `pleroma.image_sizes`, optionally as AVIF (`avif`); the Docker image adds `vips-heif` for it.
diff --git a/config/config.exs b/config/config.exs
index f4f6e9764..3819c341f 100644
--- a/config/config.exs
+++ b/config/config.exs
@@ -501,7 +501,10 @@ config :pleroma, :media_preview_proxy,
thumbnail_max_height: 600,
image_quality: 85,
min_content_length: 100 * 1024,
- image_sizes: [960, 1600, 2560]
+ image_sizes: [960, 1600, 2560],
+ avif: false,
+ avif_quality: 55,
+ avif_effort: 3
config :pleroma, :shout,
enabled: true,
diff --git a/config/description.exs b/config/description.exs
index 0902c0873..c1c87dd6a 100644
--- a/config/description.exs
+++ b/config/description.exs
@@ -1670,6 +1670,25 @@ config :pleroma, :config_description, [
description:
"Sizes (the longer side, in pixels) of image previews besides the thumbnail, offered to clients so they can show an image fitting the screen instead of the original. Only these sizes are served. An empty list turns them off.",
suggestions: [[960, 1600, 2560]]
+ },
+ %{
+ key: :avif,
+ type: :boolean,
+ description:
+ "Serve the image sizes as AVIF to clients that accept it (about 40% smaller than JPEG at the same quality, but several times slower to encode: put a cache in front). Needs a libvips with libheif and an AV1 encoder (the Docker image has them); without one, the sizes stay JPEG/PNG."
+ },
+ %{
+ key: :avif_quality,
+ type: :integer,
+ description: "Quality of AVIF previews, 0 to 100.",
+ suggestions: [55]
+ },
+ %{
+ key: :avif_effort,
+ type: :integer,
+ description:
+ "CPU effort of the AVIF encoder, 0 (fastest) to 9 (smallest files). Each step up costs much more time for a little less size.",
+ suggestions: [3]
}
]
},
diff --git a/docs/configuration/cheatsheet.md b/docs/configuration/cheatsheet.md
index b54a5fa85..e234d5632 100644
--- a/docs/configuration/cheatsheet.md
+++ b/docs/configuration/cheatsheet.md
@@ -390,6 +390,9 @@ This section describe PWA manifest instance-specific values. Currently this opti
* `image_quality`: Quality of the output. Ranges from 0 (min quality) to 100 (max quality).
* `min_content_length`: Min content length to perform preview, in bytes. If greater than 0, media smaller in size will be served as is, without thumbnailing.
* `image_sizes`: Sizes of image previews besides the thumbnail, as the longer side in pixels (default: `[960, 1600, 2560]`). The API lists them on each image attachment (`pleroma.image_sizes`, only the sizes smaller than the image), so that clients can show an image fitting the screen instead of the original. A preview of such a size is `/proxy/preview/…?size=1600`; other sizes redirect to the thumbnail. An empty list turns them off. Every preview is made on request, so put a cache in front of `/proxy` (see the nginx example config) to resize each image only once per size.
+* `avif`: Serve the `image_sizes` as AVIF to clients whose `Accept` header includes `image/avif` (default: `false`). AVIF is about 40% smaller than JPEG at the same quality, but encoding takes several times longer, so a cache in front of `/proxy` matters even more. The responses carry `Vary: Accept`: a cache or CDN in front must honour it, or it would give one format to everyone. Needs a libvips with libheif and an AV1 encoder (on Alpine: `vips-heif`, also when compiling Vix); without one, the sizes stay JPEG or PNG. Thumbnails stay JPEG or PNG.
+* `avif_quality`: Quality of the AVIF previews, 0 to 100 (default: `55`).
+* `avif_effort`: CPU effort of the AVIF encoder, 0 (fastest) to 9 (smallest files) (default: `3`).
### Purge cache strategy
diff --git a/lib/pleroma/helpers/media_helper.ex b/lib/pleroma/helpers/media_helper.ex
index 8566ab3ea..8b4b48b73 100644
--- a/lib/pleroma/helpers/media_helper.ex
+++ b/lib/pleroma/helpers/media_helper.ex
@@ -31,16 +31,39 @@ defmodule Pleroma.Helpers.MediaHelper do
height: options.max_height,
size: :VIPS_SIZE_DOWN
) do
- if options[:format] == "png" do
- Operation.pngsave_buffer(resized, Q: options[:quality])
- else
- Operation.jpegsave_buffer(resized, Q: options[:quality], interlace: true)
+ case options[:format] do
+ "png" -> Operation.pngsave_buffer(resized, Q: options[:quality])
+ "avif" -> avif_save(resized, options)
+ _ -> Operation.jpegsave_buffer(resized, Q: options[:quality], interlace: true)
end
else
{:error, _} = error -> error
end
end
+ @doc """
+ Whether this libvips can encode AVIF. Vix makes its operations from the
+ libvips it is compiled against; the bundled one has no HEIF encoder, a
+ platform libvips with libheif and an AV1 encoder (libaom) has.
+ """
+ @spec avif_supported?() :: boolean()
+ def avif_supported? do
+ Code.ensure_loaded?(Operation) and function_exported?(Operation, :heifsave_buffer, 2)
+ end
+
+ # called through apply/3: without libheif the function does not exist
+ defp avif_save(image, options) do
+ # credo:disable-for-next-line Credo.Check.Refactor.Apply
+ apply(Operation, :heifsave_buffer, [
+ image,
+ [
+ Q: options[:quality],
+ effort: options[:effort],
+ compression: :VIPS_FOREIGN_HEIF_COMPRESSION_AV1
+ ]
+ ])
+ end
+
# Note: video thumbnail is intentionally not resized (always has original dimensions)
@spec video_framegrab(String.t()) :: {:ok, binary()} | {:error, any()}
def video_framegrab(url) do
diff --git a/lib/pleroma/web/media_proxy/media_proxy_controller.ex b/lib/pleroma/web/media_proxy/media_proxy_controller.ex
index c2a75e4bf..ccdc6c5ef 100644
--- a/lib/pleroma/web/media_proxy/media_proxy_controller.ex
+++ b/lib/pleroma/web/media_proxy/media_proxy_controller.ex
@@ -40,7 +40,9 @@ defmodule Pleroma.Web.MediaProxy.MediaProxyController do
{:ok, url} <- MediaProxy.decode_url(sig64, url64),
:ok <- MediaProxy.verify_request_path_and_url(conn, url),
{:ok, dimensions} <- preview_dimensions(conn.params["size"]) do
- handle_preview(conn, url, dimensions)
+ conn
+ |> choose_avif(conn.params["size"])
+ |> handle_preview(url, dimensions)
else
{:enabled, false} ->
send_resp(conn, 404, Conn.Status.reason_phrase(404))
@@ -73,6 +75,27 @@ defmodule Pleroma.Web.MediaProxy.MediaProxyController do
defp preview_dimensions(_size), do: :unknown_size
+ # A size (not the thumbnail) as AVIF, when it is on, this libvips can
+ # encode it and the client takes it. The answer then depends on Accept:
+ # caches must keep the formats apart.
+ defp choose_avif(conn, nil), do: conn
+
+ defp choose_avif(conn, _size) do
+ if Keyword.get(media_preview_proxy_config(), :avif, false) and
+ MediaHelper.avif_supported?() do
+ accepts_avif? =
+ conn
+ |> get_req_header("accept")
+ |> Enum.any?(&String.contains?(&1, "image/avif"))
+
+ conn
+ |> put_resp_header("vary", "accept")
+ |> assign(:preview_avif, accepts_avif?)
+ else
+ conn
+ end
+ end
+
defp handle_preview(conn, url, dimensions) do
media_proxy_url = MediaProxy.url(url)
http_client_opts = Pleroma.Config.get([:media_proxy, :proxy_opts, :http], pool: :media)
@@ -118,6 +141,30 @@ defmodule Pleroma.Web.MediaProxy.MediaProxyController do
end
end
+ defp handle_preview(
+ "image/" <> _ = content_type,
+ %{assigns: %{preview_avif: true}} = conn,
+ media_proxy_url,
+ dimensions
+ ) do
+ case avif_preview(media_proxy_url, dimensions) do
+ {:ok, binary} ->
+ conn
+ |> put_preview_response_headers(["image/avif", "preview.avif"])
+ |> send_resp(200, binary)
+
+ # an encoder that fails (a libheif without AV1): JPEG/PNG, not the
+ # original
+ _ ->
+ handle_preview(
+ content_type,
+ assign(conn, :preview_avif, false),
+ media_proxy_url,
+ dimensions
+ )
+ end
+ end
+
defp handle_preview("image/png" <> _ = _content_type, conn, media_proxy_url, dimensions) do
handle_png_preview(conn, media_proxy_url, dimensions)
end
@@ -156,6 +203,18 @@ defmodule Pleroma.Web.MediaProxy.MediaProxyController do
end
end
+ defp avif_preview(media_proxy_url, {max_width, max_height}) do
+ config = media_preview_proxy_config()
+
+ MediaHelper.image_resize(media_proxy_url, %{
+ max_width: max_width,
+ max_height: max_height,
+ quality: Keyword.get(config, :avif_quality, 55),
+ effort: Keyword.get(config, :avif_effort, 3),
+ format: "avif"
+ })
+ end
+
defp handle_jpeg_preview(conn, media_proxy_url, {thumbnail_max_width, thumbnail_max_height}) do
quality = Config.get!([:media_preview_proxy, :image_quality])
diff --git a/test/pleroma/web/media_proxy/media_proxy_controller_test.exs b/test/pleroma/web/media_proxy/media_proxy_controller_test.exs
index d9bcb8088..63bcfc2d9 100644
--- a/test/pleroma/web/media_proxy/media_proxy_controller_test.exs
+++ b/test/pleroma/web/media_proxy/media_proxy_controller_test.exs
@@ -399,6 +399,104 @@ defmodule Pleroma.Web.MediaProxy.MediaProxyControllerTest do
assert long_side(response.resp_body) == 300
end
+ @avif_skip if(Pleroma.Helpers.MediaHelper.avif_supported?(),
+ do: false,
+ else: "this libvips cannot encode AVIF"
+ )
+
+ @tag skip: @avif_skip
+ test "with AVIF on and accepted, a size comes as AVIF", %{
+ conn: conn,
+ url: url,
+ media_proxy_url: media_proxy_url
+ } do
+ clear_config([:media_preview_proxy, :image_sizes], [300])
+ clear_config([:media_preview_proxy, :avif], true)
+ mock_image(media_proxy_url, "image/jpeg", "test/fixtures/image.jpg")
+
+ response =
+ conn
+ |> put_req_header("accept", "image/avif,image/webp,*/*")
+ |> get(url <> "?size=300")
+
+ assert response.status == 200
+ assert Conn.get_resp_header(response, "content-type") == ["image/avif"]
+ assert Conn.get_resp_header(response, "vary") == ["accept"]
+ assert long_side(response.resp_body) == 300
+ end
+
+ @tag skip: @avif_skip
+ test "with AVIF on and accepted, a PNG size comes as AVIF too", %{
+ conn: conn,
+ url: url,
+ media_proxy_url: media_proxy_url
+ } do
+ clear_config([:media_preview_proxy, :image_sizes], [300])
+ clear_config([:media_preview_proxy, :avif], true)
+ mock_image(media_proxy_url, "image/png", "test/fixtures/image.png")
+
+ response =
+ conn
+ |> put_req_header("accept", "image/avif,*/*")
+ |> get(url <> "?size=300")
+
+ assert Conn.get_resp_header(response, "content-type") == ["image/avif"]
+ assert long_side(response.resp_body) == 300
+ end
+
+ @tag skip: @avif_skip
+ test "with AVIF on but not accepted, a size stays JPEG and varies on Accept", %{
+ conn: conn,
+ url: url,
+ media_proxy_url: media_proxy_url
+ } do
+ clear_config([:media_preview_proxy, :image_sizes], [300])
+ clear_config([:media_preview_proxy, :avif], true)
+ mock_image(media_proxy_url, "image/jpeg", "test/fixtures/image.jpg")
+
+ response =
+ conn
+ |> put_req_header("accept", "image/webp,*/*")
+ |> get(url <> "?size=300")
+
+ assert Conn.get_resp_header(response, "content-type") == ["image/jpeg"]
+ assert Conn.get_resp_header(response, "vary") == ["accept"]
+ end
+
+ test "with AVIF off, a size stays JPEG and does not vary", %{
+ conn: conn,
+ url: url,
+ media_proxy_url: media_proxy_url
+ } do
+ clear_config([:media_preview_proxy, :image_sizes], [300])
+ clear_config([:media_preview_proxy, :avif], false)
+ mock_image(media_proxy_url, "image/jpeg", "test/fixtures/image.jpg")
+
+ response =
+ conn
+ |> put_req_header("accept", "image/avif,*/*")
+ |> get(url <> "?size=300")
+
+ assert Conn.get_resp_header(response, "content-type") == ["image/jpeg"]
+ assert Conn.get_resp_header(response, "vary") == []
+ end
+
+ test "a thumbnail is never AVIF", %{
+ conn: conn,
+ url: url,
+ media_proxy_url: media_proxy_url
+ } do
+ clear_config([:media_preview_proxy, :avif], true)
+ mock_image(media_proxy_url, "image/jpeg", "test/fixtures/image.jpg")
+
+ response =
+ conn
+ |> put_req_header("accept", "image/avif,*/*")
+ |> get(url)
+
+ assert Conn.get_resp_header(response, "content-type") == ["image/jpeg"]
+ end
+
test "accepts a `size` only as written in the config", %{conn: conn, url: url} do
clear_config([:media_preview_proxy, :image_sizes], [800])

File Metadata

Mime Type
text/x-diff
Expires
Tue, Nov 17, 10:19 PM (17 h, 39 m)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
1839837
Default Alt Text
(14 KB)

Event Timeline