Page Menu
Home
Phorge
Search
Configure Global Search
Log In
Files
F85711336
No One
Temporary
Actions
View File
Edit File
Delete File
View Transforms
Subscribe
Award Token
Flag For Later
Size
1 KB
Referenced Files
None
Subscribers
None
View Options
diff --git a/lib/pleroma/language/language_detector.ex b/lib/pleroma/language/language_detector.ex
index b19eb4571..0be69d220 100644
--- a/lib/pleroma/language/language_detector.ex
+++ b/lib/pleroma/language/language_detector.ex
@@ -1,34 +1,42 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2022 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Language.LanguageDetector do
@words_threshold 4
def missing_dependencies do
provider = get_provider()
if provider do
provider.missing_dependencies()
else
[]
end
end
+ # Strip tags from text, etc.
+ defp prepare_text(text) do
+ text
+ |> Floki.parse_fragment!()
+ |> Floki.filter_out(".h-card, .mention, .hashtag, .u-url, .quote-inline, .recipients-inline, code, pre")
+ |> Floki.text()
+ end
+
def detect(text) do
provider = get_provider()
- {:ok, text} = text |> FastSanitize.strip_tags()
+ text = prepare_text(text)
word_count = text |> String.split(~r/\s+/) |> Enum.count()
if word_count < @words_threshold or !provider or !provider.configured? do
nil
else
provider.detect(text)
end
end
defp get_provider do
Pleroma.Config.get([__MODULE__, :provider])
end
end
File Metadata
Details
Attached
Mime Type
text/x-diff
Expires
Sat, Sep 19, 5:07 AM (1 d, 7 h)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
1768895
Default Alt Text
(1 KB)
Attached To
Mode
rPUBE pleroma-upstream
Attached
Detach File
Event Timeline
Log In to Comment