Kolab 16: Problems after Update

[Roland Kolb (IBU)]

Hello,
since the last update (4 Jul 2016) I can't send any mails from my Kolab-Server (Kolab 16 on Centos 7). My postfix-Version is: postfix.x86_64 2:2.10.1-6.el7
When I try to send a mail from the web client I found the following message in maillog: reject: DATA from localhost[::1]: 554 5.7.1 <DATA>: Data command rejected: Sender access denied;
For a testing purpose to solve the problem I changed the parameters smtpd_recipient_restrictions, submission_sender_restrictions and smtpd_sender_restrictions to permit_mynetworks. No success
smtpd_tls_auth_only = yes
transport_maps = ldap:/etc/postfix/ldap/transport_maps.cf, hash:/etc/postfix/transport
content_filter = smtp-amavis:[127.0.0.1]:10024
recipient_delimiter = +
smtpd_tls_key_file = /etc/pki/tls/private/localhost.pem
smtpd_sender_login_maps = $local_recipient_maps
local_recipient_maps = ldap:/etc/postfix/ldap/local_recipient_maps.cf
virtual_alias_maps = $alias_maps, ldap:/etc/postfix/ldap/virtual_alias_maps.cf, ldap:/etc/postfix/ldap/virtual_alias_maps_mailforwarding.cf, ldap:/etc/postfix /ldap/virtual_alias_maps_sharedfolders.cf, ldap:/etc/postfix/ldap/mailenabled_distgroups.cf, ldap:/etc/postfix/ldap/mailenabled_dynamic_distgroups.cf
submission_sender_restrictions = reject_non_fqdn_sender, check_policy_service unix:private/submission_policy, permit_sasl_authenticated, reject
submission_recipient_restrictions = check_policy_service unix:private/submission_policy, permit_sasl_authenticated, reject
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_pipelining, reject_rbl_client zen.spamhaus.org, reject_non_fqdn_recipient, reject_invalid_helo_hostname, reject_unknown_recipient_domain, reject_unauth_destination, check_policy_service unix:private/recipient_policy_incoming, permit
smtp_tls_security_level = may
submission_data_restrictions = check_policy_service unix:private/submission_policy
smtpd_tls_cert_file = /etc/pki/tls/private/localhost.pem
smtpd_tls_security_level = may
smtpd_sasl_auth_enable = yes
smtpd_sender_restrictions = permit_mynetworks, check_policy_service unix:private/sender_policy_incoming
Has somebody any idea?
Thanks in advance
Roland
Roland

[Paul Ryszka]

Hi,

same here on ubuntu 14.04 with kolab winterfell, temporary workaround is
to disable sender policy:
comment line
-o smtpd_data_restrictions=$submission_data_restrictions

under submissions section in /etc/postfix/master.cf
but that allows to send any user with any email which might be a
security concern.

Best Regards
Paul

Hello,
since the last update (4 Jul 2016) I can't send any mails from my
postfix.x86_64 2:2.10.1-6.el7
When I try to send a mail from the web client I found the following
message in maillog: reject: DATA from localhost[::1]: 554 5.7.1
<DATA>: Data command rejected: Sender access denied;
When I try to send a mail from the client (i.e. Thunderbird) I found
the following message in maillog: NOQUEUE: reject: RCPT from
business-176-094-009-211.static.arcor-ip.net
helo=<[192.168.2.102]>
For a testing purpose to solve the problem I changed the parameters
smtpd_recipient_restrictions, submission_sender_restrictions and
smtpd_sender_restrictions to permit_mynetworks. No success
smtpd_tls_auth_only = yes
transport_maps = ldap:/etc/postfix/ldap/transport_maps.cf,
hash:/etc/postfix/transport
content_filter = smtp-amavis:[127.0.0.1]:10024
recipient_delimiter = +
smtpd_tls_key_file = /etc/pki/tls/private/localhost.pem
smtpd_sender_login_maps = $local_recipient_maps
local_recipient_maps = ldap:/etc/postfix/ldap/local_recipient_maps.cf
virtual_alias_maps = $alias_maps,
ldap:/etc/postfix/ldap/virtual_alias_maps.cf,
ldap:/etc/postfix/ldap/virtual_alias_maps_mailforwarding.cf,
ldap:/etc/postfix /ldap/virtual_alias_maps_sharedfolders.cf,
ldap:/etc/postfix/ldap/mailenabled_distgroups.cf,
ldap:/etc/postfix/ldap/mailenabled_dynamic_distgroups.cf
submission_sender_restrictions = reject_non_fqdn_sender,
check_policy_service unix:private/submission_policy,
permit_sasl_authenticated, reject
submission_recipient_restrictions = check_policy_service
unix:private/submission_policy, permit_sasl_authenticated, reject
smtpd_recipient_restrictions = permit_mynetworks,
permit_sasl_authenticated, reject_unauth_pipelining,
reject_rbl_client zen.spamhaus.org <http://zen.spamhaus.org>,
reject_non_fqdn_recipient, reject_invalid_helo_hostname,
reject_unknown_recipient_domain, reject_unauth_destination,
check_policy_service unix:private/recipient_policy_incoming, permit
smtp_tls_security_level = may
submission_data_restrictions = check_policy_service
unix:private/submission_policy
smtpd_tls_cert_file = /etc/pki/tls/private/localhost.pem
smtpd_tls_security_level = may
smtpd_sasl_auth_enable = yes
smtpd_sender_restrictions = permit_mynetworks, check_policy_service
unix:private/sender_policy_incoming
Has somebody any idea?
Thanks in advance
Roland
Roland

_______________________________________________
users mailing list
https://lists.kolab.org/mailman/listinfo/users

[Jason Spangler]

This post might be inappropriate. Click to display it.

[Jeroen van Meeuwen (Kolab Systems)]

Post by Paul Ryszka
Hi,
same here on ubuntu 14.04 with kolab winterfell, temporary workaround
comment line
-o smtpd_data_restrictions=$submission_data_restrictions

I'm unable to reproduce the problem reported on a vanilla,
next-next-finish, fresh installation of Kolab Winterfell on Enterprise
Linux 7.

I'll do another attempt on Kolab 16, but here's the additional
troubleshooting and information gathering you might be able to assist
with:

- in /etc/postfix/master.cf, add '-v' to the end of the line that

Post by Paul Ryszka
submission inet n - n - -
smtpd -v

- further down in /etc/postfix/master.cf, add '-d 9' to the
invocations of /usr/libexec/postfix/kolab_smtp_access_policy. Strictly

Post by Paul Ryszka
submission_policy unix - n n - -
spawn
user=kolab-n argv=/usr/libexec/postfix/kolab_smtp_access_policy
--verify-sender --verify-recipient -d 9

This will put all sorts of very verbose information in your
/var/log/maillog, which can help us nail down to determine whether it's
an upgrade thing or a software thing or whatnot.

Kind regards,

Jeroen van Meeuwen

[Bartosiak-Jentys, Chris]

Hello everyone, I am also seeing this issue on 2 CentOS 7 Kolab16
servers since updating. Bellow is the output from my logs with Jeroen's
suggested debug logging turned on:

Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: >>> START Data
command RESTRICTIONS <<<
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: generic_checks:
name=check_policy_service
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr request
= smtpd_access_policy
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
protocol_state = DATA
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
protocol_name = ESMTP
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
client_address = ::1
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
client_name = localhost
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
reverse_client_name = localhost
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
helo_name = example.com
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr sender
= example.example-***@example.com
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
recipient = ***@example.com
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
recipient_count = 1
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
queue_id = 9AACD30833A
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
instance = 5efd.577e3657.266ae.0
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr size =
0
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
etrn_domain =
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr stress
=
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
sasl_method = LOGIN
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
sasl_username = example.example-***@example.com
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
sasl_sender =
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
ccert_subject =
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
ccert_issuer =
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
ccert_fingerprint =
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
ccert_pubkey_fingerprint =
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
encryption_protocol = TLSv1
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
encryption_cipher = ECDHE-RSA-AES256-SHA
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: send attr
encryption_keysize = 256
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,663 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: request
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,663 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: protocol_state
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,664 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: protocol_name
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,664 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: client_address
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,664 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: client_name
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,664 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: reverse_client_name
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,664 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: helo_name
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,664 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: sender
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,664 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: recipient
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,665 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: recipient_count
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,665 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: queue_id
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,665 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: instance
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,665 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: size
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,665 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: etrn_domain
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,665 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: stress
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,665 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: sasl_method
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,666 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: sasl_username
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,666 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: sasl_sender
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,666 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: ccert_subject
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,666 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: ccert_issuer
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,666 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: ccert_fingerprint
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,666 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: ccert_pubkey_fingerprint
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,667 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: encryption_protocol
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,667 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: encryption_cipher
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,667 pykolab.smtp_access_policy DEBUG [24319]:
Getting line: encryption_keysize
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,667 pykolab.smtp_access_policy DEBUG [24319]:
End of current request
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,667 pykolab.smtp_access_policy DEBUG [24319]:
Returning request
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,667 pykolab.smtp_access_policy DEBUG [24319]:
Got request instance 5efd.577e3657.266ae.0
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,667 pykolab.smtp_access_policy DEBUG [24319]:
Adding policy request to instance 5efd.577e3657.266ae.0
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,668 pykolab.smtp_access_policy DEBUG [24319]:
Request instance 5efd.577e3657.266ae.0 is in state data
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/smtpd[24322]: connect from
unknown[155.133.82.93]
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,670 sqlalchemy.engine.base.Engine INFO BEGIN
(implicit)
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,673 sqlalchemy.engine.base.Engine INFO SELECT
policy_result.id AS policy_result_id, policy_result.`key` AS
policy_result_key, policy_result.value AS policy_result_value,
policy_result.sender AS policy_result_sender, policy_result.recipient AS
policy_result_recipient, policy_result.sasl_username AS
policy_result_sasl_username, policy_result.sasl_sender AS
policy_result_sasl_sender, policy_result.created AS
policy_result_created, policy_result.data AS policy_result_data
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: FROM policy_result
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: WHERE policy_result.sasl_sender IS NULL AND policy_result.sender
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,673 sqlalchemy.engine.base.Engine INFO
('example.example-***@example.com', 'verify_sender',
'example.example-***@example.com', '***@example.com',
1467802841)
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,674 pykolab.smtp_access_policy ERROR Unhandled
exception caught: OperationalError('(OperationalError) (1054, "Unknown
column \'policy_result.data\' in \'field list\'")',)
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,679 pykolab.smtp_access_policy ERROR Traceback
(most recent call last):
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: File "/usr/libexec/postfix/kolab_smtp_access_policy", line 1659,
in <module>
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: sender_allowed
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: File "/usr/libexec/postfix/kolab_smtp_access_policy", line 1053,
in verify_sender
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: function
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: File "/usr/libexec/postfix/kolab_smtp_access_policy", line 1258,
in cache_select
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: ((int)(time.time()) - cache_expire)
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: File
"/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2320,
in all
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: return list(self)
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: File
"/usr/lib64/python2.7/site-packages/sqlalchemy/orm/query.py", line 2438,
in __iter__
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: return self._execute_and_instances(context)
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]:
private/submission_policy: wanted attribute: action
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: warning: missing
attribute action in input from private/submission_policy
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: warning: problem
talking to server private/submission_policy: Success
Jul 7 12:00:43 srv01 postfix/submission/smtpd[24317]: warning: missing
attribute action in input from private/submission_policy
Jul 7 12:00:43 srv01 postfix/submission/smtpd[24317]: warning: problem
talking to server private/submission_policy: Success


Hope it helps

Chris Bartosiak-Jentys

Post by Jeroen van Meeuwen (Kolab Systems)

Post by Paul Ryszka
Hi,
same here on ubuntu 14.04 with kolab winterfell, temporary workaround
comment line
-o smtpd_data_restrictions=$submission_data_restrictions

I'm unable to reproduce the problem reported on a vanilla,
next-next-finish, fresh installation of Kolab Winterfell on Enterprise
Linux 7.
I'll do another attempt on Kolab 16, but here's the additional
troubleshooting and information gathering you might be able to assist
- in /etc/postfix/master.cf, add '-v' to the end of the line that

Post by Paul Ryszka
submission inet n - n - -
smtpd -v

- further down in /etc/postfix/master.cf, add '-d 9' to the
invocations of /usr/libexec/postfix/kolab_smtp_access_policy. Strictly

Post by Paul Ryszka
submission_policy unix - n n - -
spawn
user=kolab-n argv=/usr/libexec/postfix/kolab_smtp_access_policy
--verify-sender --verify-recipient -d 9

This will put all sorts of very verbose information in your
/var/log/maillog, which can help us nail down to determine whether
it's an upgrade thing or a software thing or whatnot.
Kind regards,
Jeroen van Meeuwen

[Jeroen van Meeuwen (Kolab Systems)]

Post by Bartosiak-Jentys, Chris
Hello everyone, I am also seeing this issue on 2 CentOS 7 Kolab16
servers since updating. Bellow is the output from my logs with
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,674 pykolab.smtp_access_policy ERROR
Unhandled exception caught: OperationalError('(OperationalError)
(1054, "Unknown column \'policy_result.data\' in \'field list\'")',)

There's the problem.

Can you remove/drop the policy_result table from the kolab database?

Kind regards,

Jeroen van Meeuwen

[Bartosiak-Jentys, Chris]

Thanks Jeroen,

Dropping the policy_results table seems to have fixed the problem on
both servers, thank you!
Seems its automatically regenerated on the next email I send.

Any idea what could have caused this issue? I did dump the kolab
database before dropping the table as a precaution but the dump just
contains the table schema, no data was in policy_result.

Thanks,

Chris Bartosiak-Jentys

Post by Jeroen van Meeuwen (Kolab Systems)

Post by Bartosiak-Jentys, Chris
Hello everyone, I am also seeing this issue on 2 CentOS 7 Kolab16
servers since updating. Bellow is the output from my logs with
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,674 pykolab.smtp_access_policy ERROR
Unhandled exception caught: OperationalError('(OperationalError)
(1054, "Unknown column \'policy_result.data\' in \'field list\'")',)

There's the problem.
Can you remove/drop the policy_result table from the kolab database?
Kind regards,
Jeroen van Meeuwen

[Winfried Ritsch]

Thanks,

Also fixed my Kolab16, CentOS7

the new table differs in data column:

| data | blob | YES | | NULL | |

see below.

mfg
winfried

Before: MariaDB [kolab]> drop table policy_result;

+---------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| key | varchar(16) | NO | MUL | NULL | |
| value | tinyint(1) | NO | | NULL | |
| sender | varchar(64) | NO | | NULL | |
| recipient | varchar(64) | NO | | NULL | |
| sasl_username | varchar(64) | YES | | NULL | |
| sasl_sender | varchar(64) | YES | | NULL | |
| created | int(11) | NO | | NULL | |
+---------------+-------------+------+-----+---------+----------------+

After: sending an email via submission

+---------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| key | varchar(16) | NO | MUL | NULL | |
| value | tinyint(1) | NO | | NULL | |
| sender | varchar(64) | YES | | NULL | |
| recipient | varchar(64) | NO | | NULL | |
| sasl_username | varchar(64) | YES | | NULL | |
| sasl_sender | varchar(64) | YES | | NULL | |
| created | int(11) | NO | | NULL | |
| data | blob | YES | | NULL | |
+---------------+-------------+------+-----+---------+----------------+

Post by Bartosiak-Jentys, Chris
Thanks Jeroen,
Dropping the policy_results table seems to have fixed the problem on
both servers, thank you!
Seems its automatically regenerated on the next email I send.
Any idea what could have caused this issue? I did dump the kolab
database before dropping the table as a precaution but the dump just
contains the table schema, no data was in policy_result.
Thanks,
Chris Bartosiak-Jentys

Post by Jeroen van Meeuwen (Kolab Systems)

Post by Bartosiak-Jentys, Chris
Hello everyone, I am also seeing this issue on 2 CentOS 7 Kolab16
servers since updating. Bellow is the output from my logs with
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,674 pykolab.smtp_access_policy ERROR
Unhandled exception caught: OperationalError('(OperationalError)
(1054, "Unknown column \'policy_result.data\' in \'field list\'")',)

There's the problem.
Can you remove/drop the policy_result table from the kolab database?
Kind regards,
Jeroen van Meeuwen

_______________________________________________
users mailing list
https://lists.kolab.org/mailman/listinfo/users

[Bartosiak-Jentys, Chris]

Ah yes, cant believe I missed that! Thanks for the explanation Winfried.

Post by Bartosiak-Jentys, Chris
Thanks,
Also fixed my Kolab16, CentOS7
| data | blob | YES | | NULL | |
see below.
mfg
winfried
Before: MariaDB [kolab]> drop table policy_result;
+---------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| key | varchar(16) | NO | MUL | NULL | |
| value | tinyint(1) | NO | | NULL | |
| sender | varchar(64) | NO | | NULL | |
| recipient | varchar(64) | NO | | NULL | |
| sasl_username | varchar(64) | YES | | NULL | |
| sasl_sender | varchar(64) | YES | | NULL | |
| created | int(11) | NO | | NULL | |
+---------------+-------------+------+-----+---------+----------------+
After: sending an email via submission
+---------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| key | varchar(16) | NO | MUL | NULL | |
| value | tinyint(1) | NO | | NULL | |
| sender | varchar(64) | YES | | NULL | |
| recipient | varchar(64) | NO | | NULL | |
| sasl_username | varchar(64) | YES | | NULL | |
| sasl_sender | varchar(64) | YES | | NULL | |
| created | int(11) | NO | | NULL | |
| data | blob | YES | | NULL | |
+---------------+-------------+------+-----+---------+----------------+

Post by Bartosiak-Jentys, Chris
Thanks Jeroen,
Dropping the policy_results table seems to have fixed the problem on
both servers, thank you!
Seems its automatically regenerated on the next email I send.
Any idea what could have caused this issue? I did dump the kolab
database before dropping the table as a precaution but the dump just
contains the table schema, no data was in policy_result.
Thanks,
Chris Bartosiak-Jentys

Post by Jeroen van Meeuwen (Kolab Systems)

Post by Bartosiak-Jentys, Chris
Hello everyone, I am also seeing this issue on 2 CentOS 7 Kolab16
servers since updating. Bellow is the output from my logs with
Jul 7 12:00:41 srv01 postfix/submission/smtpd[24317]: input attribute
name: 2016-07-07 12:00:41,674 pykolab.smtp_access_policy ERROR
Unhandled exception caught: OperationalError('(OperationalError)
(1054, "Unknown column \'policy_result.data\' in \'field list\'")',)

There's the problem.
Can you remove/drop the policy_result table from the kolab database?
Kind regards,
Jeroen van Meeuwen

_______________________________________________
users mailing list
https://lists.kolab.org/mailman/listinfo/users

[Jeroen van Meeuwen]

Post by Bartosiak-Jentys, Chris
Thanks Jeroen,
Dropping the policy_results table seems to have fixed the problem on 
both servers, thank you!
Seems its automatically regenerated on the next email I send.
Any idea what could have caused this issue? I did dump the kolab 
database before dropping the table as a precaution but the dump just 
contains the table schema, no data was in policy_result.

Well, yeah I know what caused the issue, in part because I wrote this
stuff and in part because I'm responsible for you and others
experiencing the issue in the first place;

  - An issue in caching the policy results of users delegated the
authorization to send 'on behalf of' necessitated the introduction of
additional information, as opposed to just 'envelope sender', 'sasl
username', 'recipient' with a 'yay or nay' boolean -- this is the
policy_result.data column.

  - Adding a column in the ORM (SQLAlchemy) is easy, but offering an
automated path to migrate existing installations with a schema update
is very expensive (in time and effort and otherwise).

In our eagerness to make sure Kolab 16 functions exactly the way we
think it should until at least we do Kolab 17, I (personally) had
(mistakenly) assumed this particular fix was already in Kolab 16,
meaning no existing installation of Kolab 16 would require a schema
upgrade. All-in-all, it's therefore my mistake.

Now, as to the reason that this is cached to begin with, it is
primarily for larger hierarchies with (typically) more delegation
happening and more frequently used distribution groups, where the
number of searches against LDAP needed to establish the authorization
for one single authenticated user to use the envelope sender address
used in the message being submitted, taking in to account the
recipient(s) might have policies in place prohibiting the (envelope)
sender from sending them email, is ... well ... costly. The length of
that sentence is a testimony to that fact.

Other than that single aspect of costly searches with potentially many,
many individual recipients for a single message, this cache has no
value whatsoever. It can be discarded at any time, and recreated as an
environment returns to regular usage patterns (delays on initial
submissions implied, though).

I should note that there's a means to 'drop, lather, rinse and repeat'
in error recovery -- we've implemented such for the 'authentication
cache' already -- though rest assured this doesn't actually cache
credentials, it just caches search results (domain 'example.org' being
'dc=example,dc=org', and '***@example.org' being
'uid=doe,ou=People,dc=example,dc=org', if you will, with the password
check still being an actual bind operation against LDAP).

I suppose we could implement something similar for the SMTP Access
Policy cache (the policy_result table), which as you mentioned is
restored 'automagically' (thanks to the ORM, actually).

Kind regards,

Jeroen van Meeuwen

--
Senior Solutions Architect, Kolab Systems AG

e: vanmeeuwen at kolabsys.com
m: [phone]
w:
https://kolabsystems.com

pgp: 9342 BF08

[Bartosiak-Jentys, Chris]

Hi Jeroen,

Thank you for the detailed response, its really interesting to learn
more about Kolab's architecture and the work that goes into developing
it. I just want to say its very much appreciated, thank you to you and
all the other contributors.

I had no idea that Kolab cached expensive LDAP searches in SQL, thanks
for taking the time to explain this. I admit I had to read over your
response a few times to understand it (and still don't fully). Hopefully
a better understanding will enable myself and others to submit better
bug reports with more relevant logs in future as well as helping others
on the list.

Thank you for your time.

Kind regards,

Chris Bartosiak-Jentys

Post by Jeroen van Meeuwen

Post by Bartosiak-Jentys, Chris
Thanks Jeroen,
Dropping the policy_results table seems to have fixed the problem on 
both servers, thank you!
Seems its automatically regenerated on the next email I send.
Any idea what could have caused this issue? I did dump the kolab 
database before dropping the table as a precaution but the dump just 
contains the table schema, no data was in policy_result.

Well, yeah I know what caused the issue, in part because I wrote this
stuff and in part because I'm responsible for you and others
experiencing the issue in the first place;
  - An issue in caching the policy results of users delegated the
authorization to send 'on behalf of' necessitated the introduction of
additional information, as opposed to just 'envelope sender', 'sasl
username', 'recipient' with a 'yay or nay' boolean -- this is the
policy_result.data column.
  - Adding a column in the ORM (SQLAlchemy) is easy, but offering an
automated path to migrate existing installations with a schema update
is very expensive (in time and effort and otherwise).
In our eagerness to make sure Kolab 16 functions exactly the way we
think it should until at least we do Kolab 17, I (personally) had
(mistakenly) assumed this particular fix was already in Kolab 16,
meaning no existing installation of Kolab 16 would require a schema
upgrade. All-in-all, it's therefore my mistake.
Now, as to the reason that this is cached to begin with, it is
primarily for larger hierarchies with (typically) more delegation
happening and more frequently used distribution groups, where the
number of searches against LDAP needed to establish the authorization
for one single authenticated user to use the envelope sender address
used in the message being submitted, taking in to account the
recipient(s) might have policies in place prohibiting the (envelope)
sender from sending them email, is ... well ... costly. The length of
that sentence is a testimony to that fact.
Other than that single aspect of costly searches with potentially many,
many individual recipients for a single message, this cache has no
value whatsoever. It can be discarded at any time, and recreated as an
environment returns to regular usage patterns (delays on initial
submissions implied, though).
I should note that there's a means to 'drop, lather, rinse and repeat'
in error recovery -- we've implemented such for the 'authentication
cache' already -- though rest assured this doesn't actually cache
credentials, it just caches search results (domain 'example.org' being
'uid=doe,ou=People,dc=example,dc=org', if you will, with the password
check still being an actual bind operation against LDAP).
I suppose we could implement something similar for the SMTP Access
Policy cache (the policy_result table), which as you mentioned is
restored 'automagically' (thanks to the ORM, actually).
Kind regards,
Jeroen van Meeuwen
--
Senior Solutions Architect, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
m: [phone]
https://kolabsystems.com
pgp: 9342 BF08

[Jeroen van Meeuwen (Kolab Systems)]

Post by Bartosiak-Jentys, Chris
Hi Jeroen,
Thank you for the detailed response, its really interesting to learn
more about Kolab's architecture and the work that goes into developing
it. I just want to say its very much appreciated, thank you to you and
all the other contributors.
I had no idea that Kolab cached expensive LDAP searches in SQL, thanks
for taking the time to explain this. I admit I had to read over your
response a few times to understand it (and still don't fully).
Hopefully a better understanding will enable myself and others to
submit better bug reports with more relevant logs in future as well as
helping others on the list.

Given that you were interested, I wrote up a little something about the
caching related to authentication [1]. I hope it's well-received and
that I'm going to be able to find the time to write more such little
posts ;-)

Kind regards,

Jeroen van Meeuwen

[1]
https://kanarip.wordpress.com/2016/07/08/kolabs-authentication-cache/

[Bartosiak-Jentys, Chris]

Hi Jeroen,

Thank you so much for taking the time to write this blog post, an
interesting read and I look forward to reading more of your posts about
Kolab's architecture (should you have time to write them).

Blog bookmarked and on my reading list :-)

Thanks!

Chris.

Post by Jeroen van Meeuwen (Kolab Systems)

Post by Bartosiak-Jentys, Chris
Hi Jeroen,
Thank you for the detailed response, its really interesting to learn
more about Kolab's architecture and the work that goes into developing
it. I just want to say its very much appreciated, thank you to you and
all the other contributors.
I had no idea that Kolab cached expensive LDAP searches in SQL, thanks
for taking the time to explain this. I admit I had to read over your
response a few times to understand it (and still don't fully).
Hopefully a better understanding will enable myself and others to
submit better bug reports with more relevant logs in future as well as
helping others on the list.

Given that you were interested, I wrote up a little something about
the caching related to authentication [1]. I hope it's well-received
and that I'm going to be able to find the time to write more such
little posts ;-)
Kind regards,
Jeroen van Meeuwen
[1]
https://kanarip.wordpress.com/2016/07/08/kolabs-authentication-cache/

[gu471]

since the last update (4 Jul 2016) I can't send any mails from my
postfix.x86_64 2:2.10.1-6.el7  
When I try to send a mail from the web client I found the
554 5.7.1 <DATA>: Data command rejected: Sender access
denied;

After the latest update I've got the same issue.

it should have to do with the following line in my config:

main.cf:
#> submission_data_restrictions = check_policy_service
unix:private/submission_policy

master.cf:
submission inet n - n - - smtpd
-o cleanup_service_name=cleanup_submission
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_sasl_authenticated_header=yes
-o smtpd_client_restrictions=permit_sasl_authenticated,reject
#> -o smtpd_data_restrictions=$submission_data_restrictions
-o smtpd_recipient_restrictions=$submission_recipient_restrictions
-o smtpd_sender_restrictions=$submission_sender_restrictions

Commenting these lines out, the problem neither exist. But everybody trying
to send a mail from a mail address of my domain can send a mail, even with
invalid credentials.
Might it be a security issue or depends it on my other settings?

Three cheers for snapshots, but hopefully waiting for a fix.

[Mark Berndt]

I had the same problem after the update.
chris.bartosiak-jentys has the answer in deleting the policy_result table.

#mysql -u root -p
enter the mysql root password
<mysql prompt> connect kolab;
<mysql prompt> drop table policy_result;
<mysql prompt> \q

the table gets automatically recreated with the correct fields, no need to
restart anything.

Post by gu471

since the last update (4 Jul 2016) I can't send any mails from my
postfix.x86_64 2:2.10.1-6.el7
When I try to send a mail from the web client I found the
554 5.7.1 <DATA>: Data command rejected: Sender access
denied;

After the latest update I've got the same issue.
#> submission_data_restrictions = check_policy_service
unix:private/submission_policy
submission inet n - n - - smtpd
-o cleanup_service_name=cleanup_submission
-o syslog_name=postfix/submission
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_sasl_authenticated_header=yes
-o smtpd_client_restrictions=permit_sasl_authenticated,reject
#> -o smtpd_data_restrictions=$submission_data_restrictions
-o smtpd_recipient_restrictions=$submission_recipient_restrictions
-o smtpd_sender_restrictions=$submission_sender_restrictions
Commenting these lines out, the problem neither exist. But everybody trying
to send a mail from a mail address of my domain can send a mail, even with
invalid credentials.
Might it be a security issue or depends it on my other settings?
Three cheers for snapshots, but hopefully waiting for a fix.
_______________________________________________
users mailing list
https://lists.kolab.org/mailman/listinfo/users

[Hodosi Zoltan]

answer this problem :
http://users.kolab.narkive.com/k13CE4Zx/kolab-16-problems-after-update