# Solr and full text search

**URL:** <https://discourse.mailinabox.email/t/solr-and-full-text-search/12286>\
**Category:** Improvements\
**Created:** [August 25, 2024, 8:50pm UTC](https://discourse.mailinabox.email/t/solr-and-full-text-search/12286 "2024-08-25T20:50:36Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![zoof](https://discourse.mailinabox.email/user_avatar/discourse.mailinabox.email/zoof/32/231_2.png) [@zoof](https://discourse.mailinabox.email/u/zoof)\
**Post date:** [August 25, 2024, 8:50pm UTC](https://discourse.mailinabox.email/t/solr-and-full-text-search/12286/1 "2024-08-25T20:50:37Z")

</div>

One of the biggest complaints that I get from users is that search with Mail-in-a-Box sucks. So I decided to do some searching and ran across [this](https://discourse.mailinabox.email/t/howto-add-solr-full-text-search/6591). @tanc, I’m wondering why you run Solr as a docker container rather than building it? How much RAM/CPU does Solr use? I’m running on a Linode nanode so I’m not sure it can handle a java app. And a related question is, why does Ubuntu include a package dovecot-solr but no package for solr? Puzzling.

---

<div class="post-metadata">

**Author:** ![matidau](https://discourse.mailinabox.email/user_avatar/discourse.mailinabox.email/matidau/32/2838_2.png) [@matidau](https://discourse.mailinabox.email/u/matidau)\
**Post date:** [August 26, 2024, 5:32am UTC](https://discourse.mailinabox.email/t/solr-and-full-text-search/12286/2 "2024-08-26T05:32:58Z")

</div>

Only slightly related but [GitHub - slusarz/dovecot-fts-flatcurve: Dovecot FTS Flatcurve plugin (Xapian)](https://github.com/slusarz/dovecot-fts-flatcurve) might be a bit lighter and is the default FTS from Dovecot CE 2.4 and onwards.

---

<div class="post-metadata">

**Author:** ![latinhypercube](https://discourse.mailinabox.email/letter_avatar_proxy/v4/letter/l/46a35a/32.png) [@latinhypercube](https://discourse.mailinabox.email/u/latinhypercube)\
**Post date:** [August 26, 2024, 10:26am UTC](https://discourse.mailinabox.email/t/solr-and-full-text-search/12286/3 "2024-08-26T10:26:57Z")

</div>

SOLR is the open source solution for full text search indexing maintained by Dovecot

[https://doc.dovecot.org/configuration\_manual/fts/#searching-in-dovecot](https://doc.dovecot.org/configuration_manual/fts/#searching-in-dovecot)

Another email/server appliance (using Docker) is Cloudron. Their instructions for setting up SOLR suggest the mail service be given 3GB of memory.

[https://docs.cloudron.io/email/#full-text-search](https://docs.cloudron.io/email/#full-text-search)

I haven’t found this necessary myself so I can’t comment from actual use.

---

<div class="post-metadata">

**Author:** ![matidau](https://discourse.mailinabox.email/user_avatar/discourse.mailinabox.email/matidau/32/2838_2.png) [@matidau](https://discourse.mailinabox.email/u/matidau)\
**Post date:** [August 26, 2024, 11:42am UTC](https://discourse.mailinabox.email/t/solr-and-full-text-search/12286/4 "2024-08-26T11:42:52Z")

</div>

The dovecot doco doesn’t look like it has been updated for 2.4.

> **[What is Flatcurve? | FTS Flatcurve](https://slusarz.github.io/dovecot-fts-flatcurve/what.html)**
>
> Documentation for the fts-flatcurve Dovecot plugin

> FTS Flatcurve will become the default Dovecot Community Edition (CE) FTS driver in v2.4 ([merged into Dovecot core in April 2022](https://github.com/dovecot/core/commit/137572e77fdf79b2e8d607021667741ed3f19da1)).

> FTS Flatcurve will continued to be maintained separately in GitHub for backwards support with _ **Dovecot CE v2.3.x** _. However, it is possible that configuration and features may differ between this v2.3 code and core v2.4 code.

---

<div class="post-metadata">

**Author:** ![latinhypercube](https://discourse.mailinabox.email/letter_avatar_proxy/v4/letter/l/46a35a/32.png) [@latinhypercube](https://discourse.mailinabox.email/u/latinhypercube)\
**Post date:** [August 26, 2024, 12:00pm UTC](https://discourse.mailinabox.email/t/solr-and-full-text-search/12286/5 "2024-08-26T12:00:06Z")

</div>

Interesting to see this updated approach. Thanks.

---

<div class="post-metadata">

**Author:** ![zoof](https://discourse.mailinabox.email/user_avatar/discourse.mailinabox.email/zoof/32/231_2.png) [@zoof](https://discourse.mailinabox.email/u/zoof)\
**Post date:** [August 26, 2024, 6:54pm UTC](https://discourse.mailinabox.email/t/solr-and-full-text-search/12286/6 "2024-08-26T18:54:02Z")

</div>

Yes, very interesting. I wonder if the MIAB team would consider implementing this as a backport when 2.4 is finally released.

---

<div class="post-metadata">

**Author:** ![matidau](https://discourse.mailinabox.email/user_avatar/discourse.mailinabox.email/matidau/32/2838_2.png) [@matidau](https://discourse.mailinabox.email/u/matidau)\
**Post date:** [August 26, 2024, 7:38pm UTC](https://discourse.mailinabox.email/t/solr-and-full-text-search/12286/7 "2024-08-26T19:38:48Z")

</div>

We may be waiting until the next Ubuntu version 26.04.

---

<div class="post-metadata">

**Author:** ![KiekerJan](https://discourse.mailinabox.email/user_avatar/discourse.mailinabox.email/kiekerjan/32/2339_2.png) [@KiekerJan](https://discourse.mailinabox.email/u/KiekerJan)\
**Post date:** [August 28, 2024, 6:19pm UTC](https://discourse.mailinabox.email/t/solr-and-full-text-search/12286/8 "2024-08-28T18:19:32Z")

</div>

In the past I’ve been using this: [GitHub - grosjo/fts-xapian: Dovecot FTS plugin based on Xapian](https://github.com/grosjo/fts-xapian) However, I’m not sure it’s working. At least it produces entries in the logging 😉  
How do you see that your search has bad results?

---

<div class="post-metadata">

**Author:** ![zoof](https://discourse.mailinabox.email/user_avatar/discourse.mailinabox.email/zoof/32/231_2.png) [@zoof](https://discourse.mailinabox.email/u/zoof)\
**Post date:** [August 29, 2024, 4:39pm UTC](https://discourse.mailinabox.email/t/solr-and-full-text-search/12286/9 "2024-08-29T16:39:01Z")

</div>

Interesting – did you use the suggested configuration?

---

<div class="post-metadata">

**Author:** ![KiekerJan](https://discourse.mailinabox.email/user_avatar/discourse.mailinabox.email/kiekerjan/32/2339_2.png) [@KiekerJan](https://discourse.mailinabox.email/u/KiekerJan)\
**Post date:** [August 30, 2024, 6:08pm UTC](https://discourse.mailinabox.email/t/solr-and-full-text-search/12286/10 "2024-08-30T18:08:40Z")

</div>

I have this

```auto
root@box:/etc/dovecot/conf.d# cat 90-plugin-fts.conf
plugin {
  plugin = fts fts_xapian

  fts = xapian
  fts_xapian = partial=3 full=20 verbose=0

  fts_autoindex = yes
  fts_enforced = yes

  fts_autoindex_exclude = \Trash
  fts_autoindex_exclude2 = \Junk
  fts_autoindex_exclude3 = \Spam

  fts_decoder = decode2text
}

service indexer-worker {
        vsz_limit = 2G
}

service decode2text {
   executable = script /usr/lib/dovecot/decode2text.sh
   user = dovecot
   unix_listener decode2text {
     mode = 0666
   }
}

```

---

<div class="post-metadata">

**Author:** ![zoof](https://discourse.mailinabox.email/user_avatar/discourse.mailinabox.email/zoof/32/231_2.png) [@zoof](https://discourse.mailinabox.email/u/zoof)\
**Post date:** [August 30, 2024, 8:56pm UTC](https://discourse.mailinabox.email/t/solr-and-full-text-search/12286/11 "2024-08-30T20:56:27Z")

</div>

Hmm. I wonder if it’s actually doing anything? I after installing dovecot-fts-xapian and adding the configuration file, I ran `doveadm index -A -q \*` as the github page says and the command finished more or less instantaneously. I also don’t see the bump in CPU usage I would expect if it were indexing in the background.

---

<div class="post-metadata">

**Author:** ![KiekerJan](https://discourse.mailinabox.email/user_avatar/discourse.mailinabox.email/kiekerjan/32/2339_2.png) [@KiekerJan](https://discourse.mailinabox.email/u/KiekerJan)\
**Post date:** [August 31, 2024, 7:09am UTC](https://discourse.mailinabox.email/t/solr-and-full-text-search/12286/12 "2024-08-31T07:09:04Z")

</div>

You can set `verbose=2` and have all kinds of stuff in the log. Maybe that helps?  
But I have the same issue. I don’t know if it’s actually working. Perhaps if I do some tests on with versus without the plugin, I’ll see a difference.

> [@zoof](#):
>
> command finished more or less instantaneously

The `-q` option schedules the indexing, but it might not be run directly. Remove it to run the indexing right away.

---

<div class="post-metadata">

**Author:** ![system](https://discourse.mailinabox.email/uploads/default/original/2X/4/4d0933d13c456b7e57f296aa6cc344525b6f0c70.png) [@system](https://discourse.mailinabox.email/u/system)\
**Post date:** [October 25, 2024, 8:51pm UTC](https://discourse.mailinabox.email/t/solr-and-full-text-search/12286/13 "2024-10-25T20:51:13Z")

</div>

This topic was automatically closed after 61 days. New replies are no longer allowed.
