Spam issues and how to overcome them

[Homer Dokes]

Greetings all,

So after having employed two kolab servers for over a year now, spam is
still a huge problem.

I have found it very difficult to understand how kolab is employing the
tools to combat spam through the server and I can find nothing but
generalities when it comes to configuring for a sound anti-spam
regiment. I can find some actual configurations for earlier versions
than Kolab 3.4 but it is obvious they don't apply to 3.4 due to changes
in naming conventions, locations, etc. so while giving 'some' idea of
how to configure it... it's a guessing game on what and how it applies
to Kolab 3.4.

Allow me to review my experiences thus far and some actual issues and
results.

I have two servers running Kolab. One is in a world wide retail
environment, the other a localized service environment.

Current conditions:

Debian 7.0 (Wheesy)
Kolab 3.4 with the latest updates as of 6/11/2016
Amavis-new
Spamassissin
Razor
Pyzor
Clamav
Sieve
Utilization of Spam block lists

I have employed most of the tactics described in this document
https://lists.kolab.org/pipermail/users/2015-September/019923.html but
still have insurmountable amounts of spam making it through the system.
The two servers have been in place and fully functional for over a
year. The spam configurations have been running with the latest
definitions and settings for over 4 weeks.

I have employed bayes rules, downloaded pre-definitions for them, and
continue to use sa-learn on a daily basis through 150+ email boxes to
'learn' what is spam through the users junk boxes but it has made
absolutely no difference. The same emails keep coming through and the
spam scoring is all over the map. No consistency to it at all. Here is
the header of an example of a spam that come through many times a day,
has 100's of entries in the Junk folders of users, and yet continues to
enjoy a spam score of 1.342... far below the recommended threshold of
6.31 which is the initial default of the configuration and certainly
well below the 3.0 that I set trying to get closer to the scores the
spam emails are getting.:

Return-Path:
<2472-838548814-88-recipient=***@mail.elementdooraim.com>
Received: from mail.yadayada.com ([unix socket])
by mail (Cyrus git2.5+0-Debian-2.5~dev2015021301-0~kolab1) with LMTPA;
Sat, 11 Jun 2016 08:46:54 -0400
X-Sieve: CMU Sieve 2.4
X-Virus-Scanned: Debian amavisd-new at yadayada.com
X-Spam-Flag: NO
X-Spam-Score: 1.342
X-Spam-Level: *
X-Spam-Status: No, score=1.342 tagged_above=-10 required=3
tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
DKIM_VALID_AU=-0.1, HTML_IMAGE_ONLY_16=1.092, HTML_MESSAGE=0.001,
HTML_SHORT_LINK_IMG_2=0.001, MPART_ALT_DIFF=0.79,
RCVD_IN_BRBL_LASTEXT=1.449, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01]
autolearn=no
Received: from maria.elementdooraim.com (64-16-218-71.static.sagonet.net
[64.16.218.71])
by mail.yadayada.com (Postfix) with ESMTP id 8B8EF53C8
for <***@yadayada.com>; Sat, 11 Jun 2016 08:46:50 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=k1;
d=elementdooraim.com;
h=Mime-Version:Content-Type:Date:From:Reply-To:Subject:To:Message-ID;
i=***@elementdooraim.com; bh=Y/a1tdkArMQ8RCID0h3i1qWZh7k=;
b=QcQOWDYWhfBwK0oWa4dx1Q5kzLf9CATzFNWO4T5rk1cRPWC3UkqZb3eeQKkN+fOx+J7WrG4YrX4d
e0Lb83zfjy9ppabQL9c3Xq1TX7EURamDq2vQDgW1wlBu1XNsh9xMjXj/9MLVZ5lzqrT04i5XiAcM
aX5d/tFQyXonE9SZPPQ=
DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=k1; d=elementdooraim.com;
b=Tn1vY7j32iXCGJRBVwMVwf3cOhFw8Zi8UsrG/mJ2fEhPVotOCQFSQJVnoxEqG26G6Io9zebXzw1y
sOeFozxSf6+bmvOpMXdyYI4TSNxudp5PnKeLquFIVEh8WfvHvON8b3Hc5ZwW4cgDptLM4z1yv9NV
n66xK1DMjzeO58bQ00c=;
Mime-Version: 1.0
Content-Type: multipart/alternative;
boundary="18112c6dd97e31c483b0c78bfc6a8313"
Date: Sat, 11 Jun 2016 05:42:13 -0700
From: "x-700 Pocket Flashlight" <***@elementdooraim.com>
Reply-To: "x700 Pocket Flashlight" <***@elementdooraim.com>
Subject: DEADLY Pocket Flashlight (A Must Have)!
To: <***@yadayada.com>
Message-ID: <***@elementdooraim.com>
X-Wallace-Footer: YES

One would have thought that the range of the spam scores would start
from zero and move in a positive direction however I have actually seen
spam scores with a negative value. What IS the range of the score?
What is it's lowest point and what is it's highest point and how does it
get calculated?

I have also recognized that most of the spam comes through a previous
FQDN which, while it hasn't been used for years, we still get valid
email to this address and therefore it has been embedded for every user
in their email box set up as a secondary domain. As such I set up sieve
rules to push all emails going to that address into it's own folder for
each user, only to realize that it is only moving about 50% of the
emails addressed to that domain to the folder that was set up. The
other 50% still end up in their main inbox. How is this possible? The
sieve rule is based ONLY on the 'To:' address and there is only the
users address with the old domain in that field. How does it work 50%
of the time and 50% not?

I have a tremendous number of pissed users because they spend more time
sifting then addressing legitimate emails. I'd be better off defining
go/no go folders that when an email is placed into the 'no go' as an
example, it is blacklisted and never allowed to come through again but I
can find no information with Kolab references on how to accomplish
this. Is Kolab capable of setting up for the user a black and white
list through roundcubemail. If so can someone point me to a tutorial or
example of a configuration?

Can an administrator of Kolab look to the individual package's own
website documentation for configuration or because of the 'fit' into
Kolab 3.4 are those configurations meaningless? Example... I understand
that running spamd is NOT what you want to do in Kolab 3.4 because
Amavis-new actually contains some of the libraries of Spamassassin and
makes calls implicitly for Spamassassin features and does not work with
spamd at all. That alone seems to throw all the individual package's
documentation out the window as we are starting from the same base.

I have owned and ran an ISP for 15 years and dissolved it 18 months ago
and have used a wide variety of email server platforms. After the ISP,
I decided to take the plunge into Kolab but having administered it over
the last year I've really called into question it's viability as a sound
and easily maintained email platform. Quite the contrary, I have found
it to demand more of my time than any other platform I have used.
Should it be this way? Am I overlooking something? In the end... it is
really the lack of consistent and applicable documentation for the Kolab
environment that has made the experience so exasperating. I am certain
that the package over all can be and probably is a sound package, but if
one can not find the documentation that speaks to the uniqueness that is
Kolab, how does one come out of it with a positive take?

In the end, what I am looking for is how does kolab 'alter' the methods
of the anti-spam tools (amavis-new, spamassassin, razor, pyzor, etc),
from a wrapper and configuration standpoint, from their respective
'stand alone' configurations. Is there a kolab version specific
reference for a functional spam configuration. I am continually
surprised at what appears to be a tremendously inadequate repository of
information for Kolab (specifically 3.4) vs. the number of users the
platform has out there. I know I can't be the only one experiencing
these issues, or, is it that I just haven't found the 'holy grail'
repository of Kolab 3.4 information.

I would appreciate any assistance I can get here with this. I am to far
invested into the Kolab platform at this time to drop it and move to
something else.

Thank you,

hdokes

[Brandt - Majentis, Gerald]

On 2016-06-11 09:46, Homer Dokes wrote:
> Greetings all,
>
> So after having employed two kolab servers for over a year now, spam
> is still a huge problem.
>
> I have found it very difficult to understand how kolab is employing
> the tools to combat spam through the server and I can find nothing but
> generalities when it comes to configuring for a sound anti-spam
> regiment. I can find some actual configurations for earlier versions
> than Kolab 3.4 but it is obvious they don't apply to 3.4 due to
> changes in naming conventions, locations, etc. so while giving 'some'
> idea of how to configure it... it's a guessing game on what and how it
> applies to Kolab 3.4.
>
> Allow me to review my experiences thus far and some actual issues and
> results.
>
> I have two servers running Kolab. One is in a world wide retail
> environment, the other a localized service environment.
>
> Current conditions:
>
> Debian 7.0 (Wheesy)
> Kolab 3.4 with the latest updates as of 6/11/2016
> Amavis-new
> Spamassissin
> Razor
> Pyzor
> Clamav
> Sieve
> Utilization of Spam block lists
>
> I have employed most of the tactics described in this document
> https://lists.kolab.org/pipermail/users/2015-September/019923.html but
> still have insurmountable amounts of spam making it through the
> system. The two servers have been in place and fully functional for
> over a year. The spam configurations have been running with the
> latest definitions and settings for over 4 weeks.
>
> I have employed bayes rules, downloaded pre-definitions for them, and
> continue to use sa-learn on a daily basis through 150+ email boxes to
> 'learn' what is spam through the users junk boxes but it has made
> absolutely no difference. The same emails keep coming through and the
> spam scoring is all over the map. No consistency to it at all. Here
> is the header of an example of a spam that come through many times a
> day, has 100's of entries in the Junk folders of users, and yet
> continues to enjoy a spam score of 1.342... far below the recommended
> threshold of 6.31 which is the initial default of the configuration
> and certainly well below the 3.0 that I set trying to get closer to
> the scores the spam emails are getting.:
>
> Return-Path:
> <2472-838548814-88-recipient=***@mail.elementdooraim.com>
> Received: from mail.yadayada.com ([unix socket])
> by mail (Cyrus git2.5+0-Debian-2.5~dev2015021301-0~kolab1) with
> LMTPA;
> Sat, 11 Jun 2016 08:46:54 -0400
> X-Sieve: CMU Sieve 2.4
> X-Virus-Scanned: Debian amavisd-new at yadayada.com
> X-Spam-Flag: NO
> X-Spam-Score: 1.342
> X-Spam-Level: *
> X-Spam-Status: No, score=1.342 tagged_above=-10 required=3
> tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
> DKIM_VALID_AU=-0.1, HTML_IMAGE_ONLY_16=1.092, HTML_MESSAGE=0.001,
> HTML_SHORT_LINK_IMG_2=0.001, MPART_ALT_DIFF=0.79,
> RCVD_IN_BRBL_LASTEXT=1.449, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01]
> autolearn=no
> Received: from maria.elementdooraim.com
> (64-16-218-71.static.sagonet.net
> [64.16.218.71])
> by mail.yadayada.com (Postfix) with ESMTP id 8B8EF53C8
> for <***@yadayada.com>; Sat, 11 Jun 2016 08:46:50 -0400 (EDT)
> DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=k1;
> d=elementdooraim.com;
> h=Mime-Version:Content-Type:Date:From:Reply-To:Subject:To:Message-ID;
> i=***@elementdooraim.com; bh=Y/a1tdkArMQ8RCID0h3i1qWZh7k=;
> b=QcQOWDYWhfBwK0oWa4dx1Q5kzLf9CATzFNWO4T5rk1cRPWC3UkqZb3eeQKkN+fOx+J7WrG4YrX4d
> e0Lb83zfjy9ppabQL9c3Xq1TX7EURamDq2vQDgW1wlBu1XNsh9xMjXj/9MLVZ5lzqrT04i5XiAcM
> aX5d/tFQyXonE9SZPPQ=
> DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=k1;
> d=elementdooraim.com;
> b=Tn1vY7j32iXCGJRBVwMVwf3cOhFw8Zi8UsrG/mJ2fEhPVotOCQFSQJVnoxEqG26G6Io9zebXzw1y
> sOeFozxSf6+bmvOpMXdyYI4TSNxudp5PnKeLquFIVEh8WfvHvON8b3Hc5ZwW4cgDptLM4z1yv9NV
> n66xK1DMjzeO58bQ00c=;
> Mime-Version: 1.0
> Content-Type: multipart/alternative;
> boundary="18112c6dd97e31c483b0c78bfc6a8313"
> Date: Sat, 11 Jun 2016 05:42:13 -0700
> From: "x-700 Pocket Flashlight" <***@elementdooraim.com>
> Reply-To: "x700 Pocket Flashlight" <***@elementdooraim.com>
> Subject: DEADLY Pocket Flashlight (A Must Have)!
> To: <***@yadayada.com>
> Message-ID:
> <***@elementdooraim.com>
> X-Wallace-Footer: YES
>
> One would have thought that the range of the spam scores would start
> from zero and move in a positive direction however I have actually
> seen spam scores with a negative value. What IS the range of the
> score? What is it's lowest point and what is it's highest point and
> how does it get calculated?
>
> I have also recognized that most of the spam comes through a previous
> FQDN which, while it hasn't been used for years, we still get valid
> email to this address and therefore it has been embedded for every
> user in their email box set up as a secondary domain. As such I set
> up sieve rules to push all emails going to that address into it's own
> folder for each user, only to realize that it is only moving about 50%
> of the emails addressed to that domain to the folder that was set up.
> The other 50% still end up in their main inbox. How is this possible?
> The sieve rule is based ONLY on the 'To:' address and there is only
> the users address with the old domain in that field. How does it work
> 50% of the time and 50% not?
>
> I have a tremendous number of pissed users because they spend more
> time sifting then addressing legitimate emails. I'd be better off
> defining go/no go folders that when an email is placed into the 'no
> go' as an example, it is blacklisted and never allowed to come through
> again but I can find no information with Kolab references on how to
> accomplish this. Is Kolab capable of setting up for the user a black
> and white list through roundcubemail. If so can someone point me to a
> tutorial or example of a configuration?
>
> Can an administrator of Kolab look to the individual package's own
> website documentation for configuration or because of the 'fit' into
> Kolab 3.4 are those configurations meaningless? Example... I
> understand that running spamd is NOT what you want to do in Kolab 3.4
> because Amavis-new actually contains some of the libraries of
> Spamassassin and makes calls implicitly for Spamassassin features and
> does not work with spamd at all. That alone seems to throw all the
> individual package's documentation out the window as we are starting
> from the same base.
>
> I have owned and ran an ISP for 15 years and dissolved it 18 months
> ago and have used a wide variety of email server platforms. After the
> ISP, I decided to take the plunge into Kolab but having administered
> it over the last year I've really called into question it's viability
> as a sound and easily maintained email platform. Quite the contrary, I
> have found it to demand more of my time than any other platform I have
> used. Should it be this way? Am I overlooking something? In the
> end... it is really the lack of consistent and applicable
> documentation for the Kolab environment that has made the experience
> so exasperating. I am certain that the package over all can be and
> probably is a sound package, but if one can not find the documentation
> that speaks to the uniqueness that is Kolab, how does one come out of
> it with a positive take?
>
> In the end, what I am looking for is how does kolab 'alter' the
> methods of the anti-spam tools (amavis-new, spamassassin, razor,
> pyzor, etc), from a wrapper and configuration standpoint, from their
> respective 'stand alone' configurations. Is there a kolab version
> specific reference for a functional spam configuration. I am
> continually surprised at what appears to be a tremendously inadequate
> repository of information for Kolab (specifically 3.4) vs. the number
> of users the platform has out there. I know I can't be the only one
> experiencing these issues, or, is it that I just haven't found the
> 'holy grail' repository of Kolab 3.4 information.
>
> I would appreciate any assistance I can get here with this. I am to
> far invested into the Kolab platform at this time to drop it and move
> to something else.
>
> Thank you,
>
> hdokes
> _______________________________________________


I installed ScrollOutF1 in front of my servers.

Gerald

[Lance Charette]

On 6/11/2016 1:25 PM, Brandt - Majentis, Gerald wrote:
> On 2016-06-11 09:46, Homer Dokes wrote:
>> Greetings all,
>>
>> So after having employed two kolab servers for over a year now, spam
>> is still a huge problem.
>>
>> I have found it very difficult to understand how kolab is employing
>> the tools to combat spam through the server and I can find nothing but
>> generalities when it comes to configuring for a sound anti-spam
>> regiment. I can find some actual configurations for earlier versions
>> than Kolab 3.4 but it is obvious they don't apply to 3.4 due to
>> changes in naming conventions, locations, etc. so while giving 'some'
>> idea of how to configure it... it's a guessing game on what and how it
>> applies to Kolab 3.4.
>>
>> Allow me to review my experiences thus far and some actual issues and
>> results.
>>
>> I have two servers running Kolab. One is in a world wide retail
>> environment, the other a localized service environment.
>>
>> Current conditions:
>>
>> Debian 7.0 (Wheesy)
>> Kolab 3.4 with the latest updates as of 6/11/2016
>> Amavis-new
>> Spamassissin
>> Razor
>> Pyzor
>> Clamav
>> Sieve
>> Utilization of Spam block lists
>>
>> I have employed most of the tactics described in this document
>> https://lists.kolab.org/pipermail/users/2015-September/019923.html but
>> still have insurmountable amounts of spam making it through the
>> system. The two servers have been in place and fully functional for
>> over a year. The spam configurations have been running with the
>> latest definitions and settings for over 4 weeks.
>>
>> I have employed bayes rules, downloaded pre-definitions for them, and
>> continue to use sa-learn on a daily basis through 150+ email boxes to
>> 'learn' what is spam through the users junk boxes but it has made
>> absolutely no difference. The same emails keep coming through and the
>> spam scoring is all over the map. No consistency to it at all. Here
>> is the header of an example of a spam that come through many times a
>> day, has 100's of entries in the Junk folders of users, and yet
>> continues to enjoy a spam score of 1.342... far below the recommended
>> threshold of 6.31 which is the initial default of the configuration
>> and certainly well below the 3.0 that I set trying to get closer to
>> the scores the spam emails are getting.:
>>
>> Return-Path:
>> <2472-838548814-88-recipient=***@mail.elementdooraim.com>
>> Received: from mail.yadayada.com ([unix socket])
>> by mail (Cyrus git2.5+0-Debian-2.5~dev2015021301-0~kolab1) with
>> LMTPA;
>> Sat, 11 Jun 2016 08:46:54 -0400
>> X-Sieve: CMU Sieve 2.4
>> X-Virus-Scanned: Debian amavisd-new at yadayada.com
>> X-Spam-Flag: NO
>> X-Spam-Score: 1.342
>> X-Spam-Level: *
>> X-Spam-Status: No, score=1.342 tagged_above=-10 required=3
>> tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
>> DKIM_VALID_AU=-0.1, HTML_IMAGE_ONLY_16=1.092, HTML_MESSAGE=0.001,
>> HTML_SHORT_LINK_IMG_2=0.001, MPART_ALT_DIFF=0.79,
>> RCVD_IN_BRBL_LASTEXT=1.449, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01]
>> autolearn=no
>> Received: from maria.elementdooraim.com (64-16-218-71.static.sagonet.net
>> [64.16.218.71])
>> by mail.yadayada.com (Postfix) with ESMTP id 8B8EF53C8
>> for <***@yadayada.com>; Sat, 11 Jun 2016 08:46:50 -0400 (EDT)
>> DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=k1;
>> d=elementdooraim.com;
>> h=Mime-Version:Content-Type:Date:From:Reply-To:Subject:To:Message-ID;
>> i=***@elementdooraim.com; bh=Y/a1tdkArMQ8RCID0h3i1qWZh7k=;
>> b=QcQOWDYWhfBwK0oWa4dx1Q5kzLf9CATzFNWO4T5rk1cRPWC3UkqZb3eeQKkN+fOx+J7WrG4YrX4d
>>
>> e0Lb83zfjy9ppabQL9c3Xq1TX7EURamDq2vQDgW1wlBu1XNsh9xMjXj/9MLVZ5lzqrT04i5XiAcM
>>
>> aX5d/tFQyXonE9SZPPQ=
>> DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=k1;
>> d=elementdooraim.com;
>> b=Tn1vY7j32iXCGJRBVwMVwf3cOhFw8Zi8UsrG/mJ2fEhPVotOCQFSQJVnoxEqG26G6Io9zebXzw1y
>>
>> sOeFozxSf6+bmvOpMXdyYI4TSNxudp5PnKeLquFIVEh8WfvHvON8b3Hc5ZwW4cgDptLM4z1yv9NV
>>
>> n66xK1DMjzeO58bQ00c=;
>> Mime-Version: 1.0
>> Content-Type: multipart/alternative;
>> boundary="18112c6dd97e31c483b0c78bfc6a8313"
>> Date: Sat, 11 Jun 2016 05:42:13 -0700
>> From: "x-700 Pocket Flashlight" <***@elementdooraim.com>
>> Reply-To: "x700 Pocket Flashlight" <***@elementdooraim.com>
>> Subject: DEADLY Pocket Flashlight (A Must Have)!
>> To: <***@yadayada.com>
>> Message-ID:
>> <***@elementdooraim.com>
>> X-Wallace-Footer: YES
>>
>> One would have thought that the range of the spam scores would start
>> from zero and move in a positive direction however I have actually
>> seen spam scores with a negative value. What IS the range of the
>> score? What is it's lowest point and what is it's highest point and
>> how does it get calculated?
>>
>> I have also recognized that most of the spam comes through a previous
>> FQDN which, while it hasn't been used for years, we still get valid
>> email to this address and therefore it has been embedded for every
>> user in their email box set up as a secondary domain. As such I set
>> up sieve rules to push all emails going to that address into it's own
>> folder for each user, only to realize that it is only moving about 50%
>> of the emails addressed to that domain to the folder that was set up.
>> The other 50% still end up in their main inbox. How is this possible?
>> The sieve rule is based ONLY on the 'To:' address and there is only
>> the users address with the old domain in that field. How does it work
>> 50% of the time and 50% not?
>>
>> I have a tremendous number of pissed users because they spend more
>> time sifting then addressing legitimate emails. I'd be better off
>> defining go/no go folders that when an email is placed into the 'no
>> go' as an example, it is blacklisted and never allowed to come through
>> again but I can find no information with Kolab references on how to
>> accomplish this. Is Kolab capable of setting up for the user a black
>> and white list through roundcubemail. If so can someone point me to a
>> tutorial or example of a configuration?
>>
>> Can an administrator of Kolab look to the individual package's own
>> website documentation for configuration or because of the 'fit' into
>> Kolab 3.4 are those configurations meaningless? Example... I
>> understand that running spamd is NOT what you want to do in Kolab 3.4
>> because Amavis-new actually contains some of the libraries of
>> Spamassassin and makes calls implicitly for Spamassassin features and
>> does not work with spamd at all. That alone seems to throw all the
>> individual package's documentation out the window as we are starting
>> from the same base.
>>
>> I have owned and ran an ISP for 15 years and dissolved it 18 months
>> ago and have used a wide variety of email server platforms. After the
>> ISP, I decided to take the plunge into Kolab but having administered
>> it over the last year I've really called into question it's viability
>> as a sound and easily maintained email platform. Quite the contrary, I
>> have found it to demand more of my time than any other platform I have
>> used. Should it be this way? Am I overlooking something? In the
>> end... it is really the lack of consistent and applicable
>> documentation for the Kolab environment that has made the experience
>> so exasperating. I am certain that the package over all can be and
>> probably is a sound package, but if one can not find the documentation
>> that speaks to the uniqueness that is Kolab, how does one come out of
>> it with a positive take?
>>
>> In the end, what I am looking for is how does kolab 'alter' the
>> methods of the anti-spam tools (amavis-new, spamassassin, razor,
>> pyzor, etc), from a wrapper and configuration standpoint, from their
>> respective 'stand alone' configurations. Is there a kolab version
>> specific reference for a functional spam configuration. I am
>> continually surprised at what appears to be a tremendously inadequate
>> repository of information for Kolab (specifically 3.4) vs. the number
>> of users the platform has out there. I know I can't be the only one
>> experiencing these issues, or, is it that I just haven't found the
>> 'holy grail' repository of Kolab 3.4 information.
>>
>> I would appreciate any assistance I can get here with this. I am to
>> far invested into the Kolab platform at this time to drop it and move
>> to something else.
>>
>> Thank you,
>>
>> hdokes
>> _______________________________________________
>
>
> I installed ScrollOutF1 in front of my servers.
>
> Gerald
>
>
Hi Gerald,

Not the answer I was expecting however I'm giving it a go since the
silence is defining on a solution that utilizes the tools already
incorporated into Kolab. It appears ScrollOut is using the same tools.
Will do a comparison after the fact and see if I can actually get the
Kolab tools to work worth a damn.

Thanks again,

hdokes

[Homer Dokes]

Hi Gerald, and all,

Hopefully you see this post. It is essentially a continuation of the
"Spam issues and how to overcome them" post but with a focus on spam
tags in the headers.

When using the Kolab tools for spam I had no issues enabling header spam
tags to show up in all emails coming through. For scrolloutf1, even tho
it has the same variable set in all levels to enable header spam tags by
default.... none of the emails coming into Kolab and being delivered
have the tags. In fact, none of them have the 'Spam' inclusion in the
subject line if it is scored as possible spam.

Have you Gerald, or any other using scrolloutf1 as a front end to Kolab
had these issues as well? If so were you able to overcome them and
how? My concern now is that Kolab may be stripping them which is why I
am not seeing them but I can't imagine it would be doing that. I don't
even get any reference to an amavis-anti-virus tag which I was getting
in Kolab.

Keep in mind I have disabled spam and anti-virus filtering in Kolab
since installing the scrolloutf1 firewall.

Thanks to all for your assistance.

hdokes



>>
>> I installed ScrollOutF1 in front of my servers.
>>
>> Gerald
>>
>>
> Hi Gerald,
>
> Not the answer I was expecting however I'm giving it a go since the
> silence is defining on a solution that utilizes the tools already
> incorporated into Kolab. It appears ScrollOut is using the same
> tools. Will do a comparison after the fact and see if I can actually
> get the Kolab tools to work worth a damn.
>
> Thanks again,
>
> hdokes
>
>

[Nathanael D. Noblet]

On Sat, 2016-06-11 at 10:46 -0400, Homer Dokes wrote:
> Greetings all,
>
> So after having employed two kolab servers for over a year now, spam
> is 
> still a huge problem.

So I never found the tools Kolab used to be effective from the get go.
Fighting spam is a semi-complicated thing to do. Setting up a mail
server is something that you need to be knowledgeable about to get
right, never mind adding in spam filtering. I'll tell you however two
bits that before I started using Kolab made the biggest difference in
our spam.

#1 - Use Real time blacklists. The most effective for us is
barracuda's. Its free but you have to provide them with the IP address
your server will make requests from. Once added the amount of spam
dropped for us about 70-90% depending on the day. 

#2 - It also helps to reject by default a handful of other non-standard
type of mail

#3 - Use a greylister. A greylisting program will watch mail arriving.
If it doesn't recognize the IP/sender. It will tell postfix to send a
temporary error. It does this for some configurable amount of time. For
us its 10 minutes. This stops quite a bit of spam as well because they
don't try over and over.

So in my postfix main I have the following:
smtpd_recipient_restrictions = permit_sasl_authenticated,
        reject_invalid_hostname,
        reject_non_fqdn_sender,
        reject_non_fqdn_recipient,
        reject_unknown_recipient_domain,
        reject_unauth_pipelining,
        permit_mynetworks,
        reject_unauth_destination,
        check_client_access hash:/etc/postfix/rbl_override,
        reject_rbl_client b.barracudacentral.org,
        reject_rbl_client sbl.spamhaus.org,
        check_policy_service unix:/var/spool/postfix/postgrey/socket

#4 - We use dspam for spam filtering. Once trained with a sufficient
corpus of mail I have found it to be better than anything else. When I
found the program, it was still being developped but that has tailed
off quite a bit but it still works well for our purposes. Here you have
to know what you are doing when you set it up. It is similar to Amavis
but in my opinion works better.

Hope that helps,
-- 

Nathanael

[Gustav Spellauge]

do not forget

#0 - use postscreen, a cheap (i terms of cpu, memory and bandwith) and
fast test which efficiently blocks mails/connections from crackes pcs.

On 06/13/2016 05:07 PM, Nathanael D. Noblet wrote:
> On Sat, 2016-06-11 at 10:46 -0400, Homer Dokes wrote:
>> Greetings all,
>>
>> So after having employed two kolab servers for over a year now, spam
>> is
>> still a huge problem.
>
> So I never found the tools Kolab used to be effective from the get go.
> Fighting spam is a semi-complicated thing to do. Setting up a mail
> server is something that you need to be knowledgeable about to get
> right, never mind adding in spam filtering. I'll tell you however two
> bits that before I started using Kolab made the biggest difference in
> our spam.
>
> #1 - Use Real time blacklists. The most effective for us is
> barracuda's. Its free but you have to provide them with the IP address
> your server will make requests from. Once added the amount of spam
> dropped for us about 70-90% depending on the day.
>
> #2 - It also helps to reject by default a handful of other non-standard
> type of mail
>
> #3 - Use a greylister. A greylisting program will watch mail arriving.
> If it doesn't recognize the IP/sender. It will tell postfix to send a
> temporary error. It does this for some configurable amount of time. For
> us its 10 minutes. This stops quite a bit of spam as well because they
> don't try over and over.
>
> So in my postfix main I have the following:
> smtpd_recipient_restrictions = permit_sasl_authenticated,
> reject_invalid_hostname,
> reject_non_fqdn_sender,
> reject_non_fqdn_recipient,
> reject_unknown_recipient_domain,
> reject_unauth_pipelining,
> permit_mynetworks,
> reject_unauth_destination,
> check_client_access hash:/etc/postfix/rbl_override,
> reject_rbl_client b.barracudacentral.org,
> reject_rbl_client sbl.spamhaus.org,
> check_policy_service unix:/var/spool/postfix/postgrey/socket
>
> #4 - We use dspam for spam filtering. Once trained with a sufficient
> corpus of mail I have found it to be better than anything else. When I
> found the program, it was still being developped but that has tailed
> off quite a bit but it still works well for our purposes. Here you have
> to know what you are doing when you set it up. It is similar to Amavis
> but in my opinion works better.
>
> Hope that helps,
> --
>
> Nathanael
>
> _______________________________________________
> users mailing list
> ***@lists.kolab.org
> https://lists.kolab.org/mailman/listinfo/users
>

[Philip Trickett (List)]

Hi Homer,

I have taken a similar route to you, but I found the things I
implemented that reduced spam the most were:

Greylisting using Postgrey: http://postgrey.schweikert.ch/
https://www.howtoforge.com/greylisting_postfix_postgrey

Implementing DKIM and SPF for postfix: http://www.opendkim.org/ There
are some good howtos out there as well.

I am using Kolab on Centos 7, but it should be fairly simple to
implement, the most frustrating part is waiting for the DNS updates for
DKIM.


Hope that helps,


Phil

On 11/06/16 15:46, Homer Dokes wrote:
> Greetings all,
>
> So after having employed two kolab servers for over a year now, spam
> is still a huge problem.
>
> I have found it very difficult to understand how kolab is employing
> the tools to combat spam through the server and I can find nothing but
> generalities when it comes to configuring for a sound anti-spam
> regiment. I can find some actual configurations for earlier versions
> than Kolab 3.4 but it is obvious they don't apply to 3.4 due to
> changes in naming conventions, locations, etc. so while giving 'some'
> idea of how to configure it... it's a guessing game on what and how it
> applies to Kolab 3.4.
>
> Allow me to review my experiences thus far and some actual issues and
> results.
>
> I have two servers running Kolab. One is in a world wide retail
> environment, the other a localized service environment.
>
> Current conditions:
>
> Debian 7.0 (Wheesy)
> Kolab 3.4 with the latest updates as of 6/11/2016
> Amavis-new
> Spamassissin
> Razor
> Pyzor
> Clamav
> Sieve
> Utilization of Spam block lists
>
> I have employed most of the tactics described in this document
> https://lists.kolab.org/pipermail/users/2015-September/019923.html but
> still have insurmountable amounts of spam making it through the
> system. The two servers have been in place and fully functional for
> over a year. The spam configurations have been running with the
> latest definitions and settings for over 4 weeks.
>
> I have employed bayes rules, downloaded pre-definitions for them, and
> continue to use sa-learn on a daily basis through 150+ email boxes to
> 'learn' what is spam through the users junk boxes but it has made
> absolutely no difference. The same emails keep coming through and the
> spam scoring is all over the map. No consistency to it at all. Here
> is the header of an example of a spam that come through many times a
> day, has 100's of entries in the Junk folders of users, and yet
> continues to enjoy a spam score of 1.342... far below the recommended
> threshold of 6.31 which is the initial default of the configuration
> and certainly well below the 3.0 that I set trying to get closer to
> the scores the spam emails are getting.:
>
> Return-Path:
> <2472-838548814-88-recipient=***@mail.elementdooraim.com>
> Received: from mail.yadayada.com ([unix socket])
> by mail (Cyrus git2.5+0-Debian-2.5~dev2015021301-0~kolab1) with
> LMTPA;
> Sat, 11 Jun 2016 08:46:54 -0400
> X-Sieve: CMU Sieve 2.4
> X-Virus-Scanned: Debian amavisd-new at yadayada.com
> X-Spam-Flag: NO
> X-Spam-Score: 1.342
> X-Spam-Level: *
> X-Spam-Status: No, score=1.342 tagged_above=-10 required=3
> tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
> DKIM_VALID_AU=-0.1, HTML_IMAGE_ONLY_16=1.092, HTML_MESSAGE=0.001,
> HTML_SHORT_LINK_IMG_2=0.001, MPART_ALT_DIFF=0.79,
> RCVD_IN_BRBL_LASTEXT=1.449, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01]
> autolearn=no
> Received: from maria.elementdooraim.com (64-16-218-71.static.sagonet.net
> [64.16.218.71])
> by mail.yadayada.com (Postfix) with ESMTP id 8B8EF53C8
> for <***@yadayada.com>; Sat, 11 Jun 2016 08:46:50 -0400 (EDT)
> DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=k1;
> d=elementdooraim.com;
> h=Mime-Version:Content-Type:Date:From:Reply-To:Subject:To:Message-ID;
> i=***@elementdooraim.com; bh=Y/a1tdkArMQ8RCID0h3i1qWZh7k=;
> b=QcQOWDYWhfBwK0oWa4dx1Q5kzLf9CATzFNWO4T5rk1cRPWC3UkqZb3eeQKkN+fOx+J7WrG4YrX4d
>
> e0Lb83zfjy9ppabQL9c3Xq1TX7EURamDq2vQDgW1wlBu1XNsh9xMjXj/9MLVZ5lzqrT04i5XiAcM
>
> aX5d/tFQyXonE9SZPPQ=
> DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=k1;
> d=elementdooraim.com;
> b=Tn1vY7j32iXCGJRBVwMVwf3cOhFw8Zi8UsrG/mJ2fEhPVotOCQFSQJVnoxEqG26G6Io9zebXzw1y
>
> sOeFozxSf6+bmvOpMXdyYI4TSNxudp5PnKeLquFIVEh8WfvHvON8b3Hc5ZwW4cgDptLM4z1yv9NV
>
> n66xK1DMjzeO58bQ00c=;
> Mime-Version: 1.0
> Content-Type: multipart/alternative;
> boundary="18112c6dd97e31c483b0c78bfc6a8313"
> Date: Sat, 11 Jun 2016 05:42:13 -0700
> From: "x-700 Pocket Flashlight" <***@elementdooraim.com>
> Reply-To: "x700 Pocket Flashlight" <***@elementdooraim.com>
> Subject: DEADLY Pocket Flashlight (A Must Have)!
> To: <***@yadayada.com>
> Message-ID:
> <***@elementdooraim.com>
> X-Wallace-Footer: YES
>
> One would have thought that the range of the spam scores would start
> from zero and move in a positive direction however I have actually
> seen spam scores with a negative value. What IS the range of the
> score? What is it's lowest point and what is it's highest point and
> how does it get calculated?
>
> I have also recognized that most of the spam comes through a previous
> FQDN which, while it hasn't been used for years, we still get valid
> email to this address and therefore it has been embedded for every
> user in their email box set up as a secondary domain. As such I set up
> sieve rules to push all emails going to that address into it's own
> folder for each user, only to realize that it is only moving about 50%
> of the emails addressed to that domain to the folder that was set up.
> The other 50% still end up in their main inbox. How is this
> possible? The sieve rule is based ONLY on the 'To:' address and there
> is only the users address with the old domain in that field. How does
> it work 50% of the time and 50% not?
>
> I have a tremendous number of pissed users because they spend more
> time sifting then addressing legitimate emails. I'd be better off
> defining go/no go folders that when an email is placed into the 'no
> go' as an example, it is blacklisted and never allowed to come through
> again but I can find no information with Kolab references on how to
> accomplish this. Is Kolab capable of setting up for the user a black
> and white list through roundcubemail. If so can someone point me to a
> tutorial or example of a configuration?
>
> Can an administrator of Kolab look to the individual package's own
> website documentation for configuration or because of the 'fit' into
> Kolab 3.4 are those configurations meaningless? Example... I
> understand that running spamd is NOT what you want to do in Kolab 3.4
> because Amavis-new actually contains some of the libraries of
> Spamassassin and makes calls implicitly for Spamassassin features and
> does not work with spamd at all. That alone seems to throw all the
> individual package's documentation out the window as we are starting
> from the same base.
>
> I have owned and ran an ISP for 15 years and dissolved it 18 months
> ago and have used a wide variety of email server platforms. After the
> ISP, I decided to take the plunge into Kolab but having administered
> it over the last year I've really called into question it's viability
> as a sound and easily maintained email platform. Quite the contrary, I
> have found it to demand more of my time than any other platform I have
> used. Should it be this way? Am I overlooking something? In the
> end... it is really the lack of consistent and applicable
> documentation for the Kolab environment that has made the experience
> so exasperating. I am certain that the package over all can be and
> probably is a sound package, but if one can not find the documentation
> that speaks to the uniqueness that is Kolab, how does one come out of
> it with a positive take?
>
> In the end, what I am looking for is how does kolab 'alter' the
> methods of the anti-spam tools (amavis-new, spamassassin, razor,
> pyzor, etc), from a wrapper and configuration standpoint, from their
> respective 'stand alone' configurations. Is there a kolab version
> specific reference for a functional spam configuration. I am
> continually surprised at what appears to be a tremendously inadequate
> repository of information for Kolab (specifically 3.4) vs. the number
> of users the platform has out there. I know I can't be the only one
> experiencing these issues, or, is it that I just haven't found the
> 'holy grail' repository of Kolab 3.4 information.
>
> I would appreciate any assistance I can get here with this. I am to
> far invested into the Kolab platform at this time to drop it and move
> to something else.
>
> Thank you,
>
> hdokes
> _______________________________________________
> users mailing list
> ***@lists.kolab.org
> https://lists.kolab.org/mailman/listinfo/users

[Lance Charette]

On 6/13/2016 11:17 AM, Philip Trickett (List) wrote:
> Hi Homer,
>
> I have taken a similar route to you, but I found the things I
> implemented that reduced spam the most were:
>
> Greylisting using Postgrey: http://postgrey.schweikert.ch/
> https://www.howtoforge.com/greylisting_postfix_postgrey
>
> Implementing DKIM and SPF for postfix: http://www.opendkim.org/ There
> are some good howtos out there as well.
>
> I am using Kolab on Centos 7, but it should be fairly simple to
> implement, the most frustrating part is waiting for the DNS updates
> for DKIM.
>
>
> Hope that helps,
>
>
> Phil

Thanks a bunch Phil and Nathanael for your replies.

I too had been using greylisting and spf which helped considerably
however it wasn't near enough for the amount of spam our accounts are
getting.

I was hoping to find some information on how I could set up black &
white lists that could be contributed to by each email user within the
Roundcube mail client but haven't seen anything yet. As I indicated in
the beginning of this tread, I owned and operated an ISP for over 15
years and have used a wide variety of email servers and separate
anti-spam servers as well, all set up and configured within so I have a
pretty good handle on the do's and don'ts and have done so on both
Windows based and Linux based platforms. This is the first time however
where I have used an 'environment' that takes so many tools that are
independently otherwise off the shelf and tries to meld them all into
one. It's a far cry different than just using an email server and a
separate anti-spam package... i.e. like spamassassin on it's own.

In the Kolab environment you have Kolab wrapped around everything,
amavis wrapped about spamassissin and so on and so on and it's the lack
of a well documented 'strategy' that makes it difficult to know
(understand) how one affects the other, etc. A solid block diagram of
how all the pieces fit into the puzzle would be a great start. Solid
examples of actual configuration files for a particular Kolab version
would also help a lot.

I understand that this is a 'community' effort but one HAS to believe
that the primary retail side of Kolab has already worked these issues
out and could reciprocate in the reverse direction given they benefit
from the community as they do.

Once satisfied with a working environment... and one that doesn't
require administration on a daily basis... I will post my examples for
others in 'our boat' to have something to start from. I have set up two
dedicated ScrollOutF1 vm servers, each on the same vm server the Kolab
resides on respectively and have setup virtual networking within for the
ScrollOutF1 to talk directly to the Kolab environment eliminating the
additional load on the physical network. That's working very well.
Also, as ScrollOutF1 is using many of the same tools already embedded in
Kolab, I'm actually hoping to take the settings once defined to our
satisfaction in ScrollOutF1 and migrate them to the Kolab equivalents
and ultimately take ScrollOutF1 out of the picture.

Again, thanks everyone and I will continue to push on and contribute as
it becomes apparent I have a well working environment... which I hope to
be soon. I have users that may exterminate me if I don't.

hdokes

[Carpenter, Troy]

On 2016-06-14 04:00 PM, Lance Charette wrote:
> On 6/13/2016 11:17 AM, Philip Trickett (List) wrote:
>> Hi Homer,
>>
>> I have taken a similar route to you, but I found the things I
>> implemented that reduced spam the most were:
>>
>> Greylisting using Postgrey: http://postgrey.schweikert.ch/
>> https://www.howtoforge.com/greylisting_postfix_postgrey
>>
>> Implementing DKIM and SPF for postfix: http://www.opendkim.org/ There
>> are some good howtos out there as well.
>>
>> I am using Kolab on Centos 7, but it should be fairly simple to
>> implement, the most frustrating part is waiting for the DNS updates
>> for DKIM.
>>
>>
>> Hope that helps,
>>
>>
>> Phil
>
> Thanks a bunch Phil and Nathanael for your replies.
>
> I too had been using greylisting and spf which helped considerably
> however it wasn't near enough for the amount of spam our accounts are
> getting.
>
> I was hoping to find some information on how I could set up black &
> white lists that could be contributed to by each email user within the
> Roundcube mail client but haven't seen anything yet. As I indicated
> in the beginning of this tread, I owned and operated an ISP for over
> 15 years and have used a wide variety of email servers and separate
> anti-spam servers as well, all set up and configured within so I have
> a pretty good handle on the do's and don'ts and have done so on both
> Windows based and Linux based platforms. This is the first time
> however where I have used an 'environment' that takes so many tools
> that are independently otherwise off the shelf and tries to meld them
> all into one. It's a far cry different than just using an email
> server and a separate anti-spam package... i.e. like spamassassin on
> it's own.
>
> In the Kolab environment you have Kolab wrapped around everything,
> amavis wrapped about spamassissin and so on and so on and it's the
> lack of a well documented 'strategy' that makes it difficult to know
> (understand) how one affects the other, etc. A solid block diagram of
> how all the pieces fit into the puzzle would be a great start. Solid
> examples of actual configuration files for a particular Kolab version
> would also help a lot.
>
> I understand that this is a 'community' effort but one HAS to believe
> that the primary retail side of Kolab has already worked these issues
> out and could reciprocate in the reverse direction given they benefit
> from the community as they do.
>
> Once satisfied with a working environment... and one that doesn't
> require administration on a daily basis... I will post my examples for
> others in 'our boat' to have something to start from. I have set up
> two dedicated ScrollOutF1 vm servers, each on the same vm server the
> Kolab resides on respectively and have setup virtual networking within
> for the ScrollOutF1 to talk directly to the Kolab environment
> eliminating the additional load on the physical network. That's
> working very well. Also, as ScrollOutF1 is using many of the same
> tools already embedded in Kolab, I'm actually hoping to take the
> settings once defined to our satisfaction in ScrollOutF1 and migrate
> them to the Kolab equivalents and ultimately take ScrollOutF1 out of
> the picture.
>
> Again, thanks everyone and I will continue to push on and contribute
> as it becomes apparent I have a well working environment... which I
> hope to be soon. I have users that may exterminate me if I don't.
>
> hdokes
> ***@lists.kolab.org



I'm a little late to this thread...but according to my logs, the
following smtpd_recipient_restrictions line in my postfix main.cf goes a
long way to stopping quite a bit of SPAM:

smtpd_recipient_restrictions = permit_mynetworks,
permit_sasl_authenticated,
reject_unauth_destination,
reject_invalid_hostname,
reject_unauth_pipelining,
reject_non_fqdn_recipient,
reject_unknown_recipient_domain,
reject_invalid_helo_hostname,
check_policy_service
unix:private/recipient_policy_incoming,
reject_rbl_client zen.spamhaus.org,
reject_rbl_client dbsbl.sorbs.net,
reject_rbl_client bl.spamcop.net,
reject_rbl_client rhsbl.sorbs.net,
permit

Obviously the reject_rbl_client is the section that does the most. I
haven't updated that in a while, so I make no claims as to which of
those services work except for zen.spamhaus.org and bl.spamcop.net, both
of which I've seen in my recent logs as being used to block.

For items that get through that, spamassassin still catches quite a bit.
It tags and a sieve script moves the email to the Spam directory if the
score is low enough; otherwise if the score is high, Amavisd shunts it
to a quarantine database with a web interface for users to release if
necessary.

The only thing I don't have a good handle on is training the Bayesian
database...but I only have about 10 users on the system right now.

Troy

[Andrea Soliva]

This post might be inappropriate. Click to display it.