Rate limit e-mail providers
If you're like me and need to manage large amounts of emails, one thing that must be done is limiting the speed email providers like yahoo.com, hotmail.com, live.com, gmail.com, aol.com get your email.
I don't go into much detail here but this is the setup I've researched and currently use.
First we edit main.cf and go to line 561 (i just like to keep things together)
vi /etc/postfix/main.cf
Add the following lines into main.cf
transport_maps = hash:/etc/postfix/transport
smtpslow_destination_concurrency_limit = 2
smtpslow_destination_recipient_limit = 15
smtpslow_destination_rate_delay = 2s
smtpslow_destination_concurrency_failed_cohort_limit = 2
Edit the transport file and add the domains we'd like to slow sending with.
vi /etc/postfix/transport
yahoo.com smtpslow:
gmail.com smtpslow:
hotmail.com smtpslow:
aol.com smtpslow:
comcast.com smtpslow:
live.com smtpslow:
msn.com smtpslow:
sbcglobal.net smtpslow:
verizon.net smtpslow:
bellsouth.net smtpslow:
yahoo.ca smtpslow:
cox.net smtpslow:
ymail.com smtpslow:
Go to the postfix directory and postmap(hash) the transport file
cd /etc/postfix && /usr/local/sbin/postmap transport
Now we need to change the following in master.cf.
vi /etc/postfix/master.cf
Make sure you have space or tabs before -o as it will not work if you don't.
smtpslow unix - - n - - smtp
-o smtp_helo_timeout=5
-o smtp_connect_timeout=5
Restart postfix and we're done.
/usr/local/sbin/postfix reload
setup jails on freebsd
we'll be creating 4 jails + main host (5 machines in total)
First, lets create some folders where the jails will be stored.
mkdir /usr/jails && mkdir /usr/jails/mail1 && mkdir /usr/jails/mail2 && mkdir /usr/jails/mail3 && mkdir /usr/jails/mail4
then we add some settings to rc.conf
# vi /etc/rc.conf
Add these line right after (remember, every machine has a differnt network card reference (bge0)). Change it to your machines network card.
ifconfig_bge0="inet 192.168.1.157 netmask 255.255.255.0"
ifconfig_bge0_alias0="inet 192.168.1.211 netmask 0xfffffff8"
ifconfig_bge0_alias1="inet 192.168.1.212 netmask 0xfffffff8"
ifconfig_bge0_alias2="inet 192.168.1.213 netmask 0xfffffff8"
ifconfig_bge0_alias3="inet 192.168.1.214 netmask 0xfffffff8"
if you don't want to reboot, you'll also have to setup the alias now so you can ssh in later...
ifconfig bge0 inet alias 192.168.1.211 netmask 255.255.255.255
ifconfig bge0 inet alias 192.168.1.212 netmask 255.255.255.255
ifconfig bge0 inet alias 192.168.1.213 netmask 255.255.255.255
ifconfig bge0 inet alias 192.168.1.214 netmask 255.255.255.255
near the bottom of the file, add following and makre sure you change the IP's & paths if different than above.
#only have 5 IP's. 1 for the main server and 4 for the jails
#jail_list="mail1 mail2 mail3 mail4"
jail_enable="YES"
jail_list="mail1 mail2 mail3 mail4"
jail_set_hostname_allow="YES"
jail_mail1_rootdir="/usr/jails/mail1"
jail_mail1_hostname="mail1.example.com"
jail_mail1_ip="192.168.1.231"
jail_mail1_start="/bin/sh /etc/rc"
jail_mail1_devfs_enable="YES"
jail_mail1_devfs_ruleset="devfsrules_jail"
jail_mail2_rootdir="/usr/jails/mail2"
jail_mail2_hostname="mail2.example.com"
jail_mail2_ip="192.168.1.232"
jail_mail2_start="/bin/sh /etc/rc"
jail_mail2_devfs_enable="YES"
jail_mail2_devfs_ruleset="devfsrules_jail"
jail_mail3_rootdir="/usr/jails/mail3"
jail_mail3_hostname="mail3.example.com"
jail_mail3_ip="192.168.1.233"
jail_mail3_start="/bin/sh /etc/rc"
jail_mail3_devfs_enable="YES"
jail_mail3_devfs_ruleset="devfsrules_jail"
jail_mail4_rootdir="/usr/jails/mail4"
jail_mail4_hostname="mail4.example.com"
jail_mail4_ip="192.168.1.234"
jail_mail4_start="/bin/sh /etc/rc"
jail_mail4_devfs_enable="YES"
jail_mail4_devfs_ruleset="devfsrules_jail"
------------------------------------------
Make the virtual system
cd /usr/src
make world DESTDIR=/usr/jails/mail1
if you get this error "make: don't know how to make world. Stop"
it's because you don't have to source tree installed...
look at cvs-supfile blog post to know how to do it...
cd etc
make distribution DESTDIR=/usr/jails/mail1
cd /usr/jails/mail1
ln -sf /dev/null kernel
touch /usr/jails/mail1/etc/fstab
cat /etc/resolv.conf > /usr/jails/mail1/etc/resolv.conf
mount -t procfs proc /usr/jails/mail1/proc
mount -t devfs dev /usr/jails/mail1/dev
mkdir /usr/jails/mail1/usr/ports
mount_nullfs /usr/ports /usr/jails/mail1/usr/ports
Once the first jail is created, we can just copy jails and not have to redo "make world" which takes awhile.
----------------------------------------------
Configuration of the jail
Start the jail for the first time
# /etc/rc.d/jail start mail1
If you try to SSH into the jail, you will end up in the main machine and not in the jail.
To fix this add the following line to /etc/rc.conf
rpcbind_enable="NO"
vi /etc/ssh/sshd_config
Port 22
Protocol 2
ListenAddress 192.168.1.157
Now we jump into the jail for some 1st step configurations
# jail /usr/jails/mail1 mail1.example.com 192.168.1.231 /bin/sh
now you are inside the jail:
change the root password
# passwd
add a new user and invite the user to the wheel group (so you can su to root)
# adduser
So we can ssh into the jail
# echo 'sshd_enable="YES"' >> /etc/rc.conf
install bash, change the users shell to bash
if you are in a jail and ports doesn't exit you have to remount the ports Filesystem
# cd /usr/ports/shells/bash
# make clean; make install clean
# chsh
change the shell to: /usr/local/bin/bash
Host environment syslogd
This entry in /etc/rc.conf tells syslogd to not listen on any IP address.
syslogd_flags="-ss"
That allows syslogd to run in both the host and the jail environments.
Host environment inetd
This entry in /etc/rc.conf tells inetd to listen on a specific IP address. This address is that of the host environment:
inetd_flags="-wW -C 60 -a 192.168.0.100"
You should note that the first part of the above flags is from /etc/defaults/rc.conf:
inetd_flags="-wW -C 60" # Optional flags to inetd
jail_enable="YES"
jail_list="mail_1"
jail_set_hostname_allow="YES"
jail_mail_1_rootdir="/usr/jails/mail_1"
jail_mail_1_hostname="mail_1.example.com"
jail_mail_1_ip="192.168.1.202" jail_mail_1_start="/bin/sh /etc/rc"
jail_mail_1_devfs_enable="YES"
jail_mail_1_devfs_ruleset="devfsrules_jail"
if you forget root password
jail /dumpdrive500/jails/devbox2 devbox2 192.168.1.152 /usr/bin/passwd root
------------------------------------------------
Clone the Jails
mkdir /usr/jails/mail2
cd /usr/jails/mail1
tar -cpf - . | tar -C /usr/jails/mail2 -xpf -
Mount all the needed directories for the cloned jails
mount -t procfs proc /usr/jails/mail1/proc
mount -t devfs dev //usr/jails/mail1/dev
mount_nullfs /usr/ports /usr/jails/mail1/usr/ports
vi /etc/rc.conf
added in -------------
jail_devbox5_rootdir="/usr/jails/mail1"
jail_devbox5_hostname="devbox5.example.com"
jail_devbox5_ip="192.168.1.155"
jail_devbox5_start="/bin/sh /etc/rc"
jail_devbox5_devfs_enable="YES"
jail_devbox5_devfs_ruleset="devfsrules_jail"
/etc/rc.d/jail start devbox5
ref: http://www.freebsd.org/doc/en/books/handbook/jails-build.html [[BR]]
ref: http://www.freebsddiary.org/jail-6.php
freebsd cyrus sasl2 saslauthd
Here is a step by step setup of what I did to get cyrus sasl2 saslauthd working on Freebsd
You'll notice that i reinstall postfix due to the fact that I needed it compiled with sasl2 which I didn't have.
# cd /usr/ports/security/cyrus-sasl2-saslauthd
# make install clean
# cd /usr/ports/mail/postfix
# make config (make sure you check on cyrus-sasl2-saslauthd)
# make
# make FORCE_PKG_REGISTER=yes install clean
now you edit /etc/rc.conf and add the following lines near the top of the file.
postfix_enable="YES"
saslauthd_enable="YES"
#If you not need sendmail anymore, please add in your rc.conf:
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"
# And you can disable some sendmail specific daily maintenance routines in your system
# If /etc/periodic.conf does not exist please create it and add those values.
/etc/periodic.conf file:
daily_clean_hoststat_enable="NO"
daily_status_mail_rejects_enable="NO"
daily_status_include_submit_mailq="NO"
daily_submit_queuerun="NO"
After installing saslauthd
/usr/local/etc/rc.d/saslauthd start
vi /etc/postfix/main.cf
Add this to the end...
# sasl config
broken_sasl_auth_clients = yes
smtpd_sasl_auth_enable = yes
smtpd_sasl_local_domain =
smtpd_sender_restrictions = permit_mynetworks, permit_sasl_authenticated
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination
/usr/local/sbin/postfix stop && /usr/local/sbin/postfix start
/usr/local/sbin/saslpasswd2 -c -u domain.com -a smtpauth usernameeee
/usr/local/etc/rc.d/saslauthd restart
how to add a new munin node with freebsd
Here is what I did to add a new munin node in freebsd to an existing munin server.
cd /usr/ports/sysutils/munin-node/
make install clean
When asked about the logs, answer yes. We want munin to rotate the logs.
Would you like me to set up log rotation [y]?
After the munin install this is what we see and need to do.
********************************************************************
Unless this file already existed, a sample configuration file
has been placed in /usr/local/etc/munin/munin-node.conf.Please edit it according to your needs.
********************************************************************
The Munin client will *not* be started automatically. To allow it
to start, put this line in /etc/rc.conf:munin_node_enable="YES"
Then, it will be started on the next boot. If this line is already
present, the client will be started now. Otherwise, edit
/etc/rc.conf and execute this command:/usr/local/etc/rc.d/munin-node start
********************************************************************
Lets do the easy stuff first for munin.
vi /etc/rc.conf
add munin_node_enable="YES" somewhere in the file. I like to keep all the enable options together
munin_node_enable="YES"
The plugins directory in freebsd is empty, so for munin node to work we need to add symbolic links.
cd /usr/local/etc/munin/plugins
ln -s /usr/local/share/munin/plugins/cpu cpu
ln -s /usr/local/share/munin/plugins/df df
ln -s /usr/local/share/munin/plugins/df_inode df_inode
ln -s /usr/local/share/munin/plugins/if_errcoll_ if_errcoll_fxp0
ln -s /usr/local/share/munin/plugins/if_ if_fxp0
ln -s /usr/local/share/munin/plugins/load load
ln -s /usr/local/share/munin/plugins/memory memory
ln -s /usr/local/share/munin/plugins/netstat netstat
ln -s /usr/local/share/munin/plugins/ntp_offset ntp_offset
ln -s /usr/local/share/munin/plugins/open_files open_files
ln -s /usr/local/share/munin/plugins/processes processes
ln -s /usr/local/share/munin/plugins/sendmail_mailqueue sendmail_mailqueue
ln -s /usr/local/share/munin/plugins/sendmail_mailstats sendmail_mailstats
ln -s /usr/local/share/munin/plugins/sendmail_mailtraffic sendmail_mailtraffic
ln -s /usr/local/share/munin/plugins/swap swap
ln -s /usr/local/share/munin/plugins/vmstat vmstat
Next, lets look at the conf file
vi /usr/local/etc/munin/munin-node.conf
update the allow line to the IP address of the munin server
allow ^192\.168\.1\.105$
everything else is good, save and exit
for the munin server
add the IP address of that machine
vi /usr/local/etc/munin/munin.conf
Add this to the end of munin.conf
[server1.domain.com]
address 192.168.1.203
use_node_name yes
notify_alias server1load.notify_alias load
df.notify_alias df
# /
df._dev_mirror_gm0s1a.warning :85
df._dev_mirror_gm0s1a.critical :90# /tmp
df._dev_mirror_gm0s1e.warning :80
df._dev_mirror_gm0s1e.critical :90# /usr
df._dev_mirror_gm0s1f.warning :80
df._dev_mirror_gm0s1f.critical :90# /var
df._dev_mirror_gm0s1d.warning :80
df._dev_mirror_gm0s1d.critical :90
If you haven't done so you can start munin-node from
cd /usr/local/etc/rc.d/
./munin-node start
If munin node doesn't update after 5..10 minutes look at the munin main log to find any errors that you may have.
cd /var/log/munin-main
vi munin-update.log
mod_wsgi apache freebsd python
The version of Apache on older versions of FreeBSD is not compiled with support for multithreading but Python itself is. Normally this would be where Apache 1.3 is being used. The result is that when starting up Apache with mod_wsgi, an error similar to the following can occur:
Syntax error on line 37 of /usr/local/apache2/conf/httpd.conf: \
Cannot load /usr/local/apache2/modules/mod_wsgi.so into server: \
/usr/local/apache2/modules/mod_wsgi.so: Undefined symbol \
"pthread_attr_init"
You may also get this Error with apache 1.3
Syntax error on line 244 of /usr/local/etc/apache/httpd.conf:
Cannot load /usr/local/libexec/apache/mod_wsgi.so into server: \
/usr/local/lib/libpython2.4.so: Undefined symbol "pthread_attr_destroy"
The solution is to force Apache to preload the reentrant version of the C runtime library which contains the POSIX thread library functions. This can be done by modifying the 'envvars' script, if it exists and is used, contained in the same directory as where the Apache 'httpd' binary is installed, or the Apache startup script, and add the lines:
LD_PRELOAD=/usr/lib/libc_r.so
export LD_PRELOAD
Step 1.
Set the library to system environment.
# LD_PRELOAD=/usr/lib/libc_r.so
# export LD_PRELOAD
Step 2.
Stop apache then start it up again
root@host rc.d # ./apache stop
Stopping apache.
Waiting for PIDS: 35117.
root@host rc.d # ./apache start
Starting apache.
root@host rc.d #
sqlite error trac install
While trying to install trac on freebsd 7.0 I had an error with sqlite.
The problem was that I had an old version already installed, but the system didn't tell me that!
# cd /usr/ports/www/trac
# make install clean
Installed /usr/local/lib/python2.5/site-packages/pytz-2009r-py2.5.egg
===> Registering installation for py25-pytz-2009r
===> Returning to build of trac-0.11.5
===> trac-0.11.5 depends on file: /usr/local/lib/python2.5/site-packages/_sqlite3.so - not found
===> Verifying install for /usr/local/lib/python2.5/site-packages/_sqlite3.so in /usr/ports/databases/py-sqlite3
===> Vulnerability check disabled, database not found
===> Extracting for py25-sqlite3-2.5.2_1
=> No MD5 checksum recorded for python/Python-2.5.2.tgz.
=> No SHA256 checksum recorded for python/Python-2.5.2.tgz.
=> No suitable checksum found for python/Python-2.5.2.tgz.
*** Error code 1
Stop in /usr/ports/databases/py-sqlite3.
*** Error code 1
Stop in /usr/ports/www/trac.
*** Error code 1
Stop in /usr/ports/www/trac.
I thought to myself, maybe I need to update the system.
# cvsup -g -L 2 /root/cvs-supfile
# make install clean
That did nothing, I get this same error. Maybe if I install it manually
cd /usr/ports/databases/py-sqlite3
root@host py-sqlite3 # make install clean
===> Vulnerability check disabled, database not found
===> Extracting for py25-sqlite3-2.5.2_1
=> No MD5 checksum recorded for python/Python-2.5.2.tgz.
=> No SHA256 checksum recorded for python/Python-2.5.2.tgz.
=> No suitable checksum found for python/Python-2.5.2.tgz.
*** Error code 1
Stop in /usr/ports/databases/py-sqlite3.
that didn't work either, so I tried
# pkg_add -R sqlite3
But the ftp server was wrong to get the correct port...
for sh you type in
setenv PACKAGESITE ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-7-stable/Latest/
for bash you update the ftp location by typing
export PACKAGESITE=ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-7-stable/Latest/
export PACKAGESITE=ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-6-stable/Latest/
# pkg_add -r trac
That worked but this is where the Error shows me I have an old version of sqlite already installed!
...
Fetching ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-7-stable/All/py26-sqlite3-2.6.4_1.tbz... Done.
pkg_add: warning: package 'py26-sqlite3-2.6.4_1' requires 'sqlite3-3.6.19', but 'sqlite3-3.5.6' is installed
...
so I upgrade my port that I do have.
# /usr/local/sbin/portupgrade -R sqlite3
---------
root@draco sqlite3 # cd /usr/ports/www/trac
root@draco trac # make install clean
Traceback (most recent call last):
File "setup.py", line 15, in
from setuptools import setup, find_packages
ImportError: No module named setuptools
*** Error code 1
Stop in /usr/ports/www/trac.
*** Error code 1
Stop in /usr/ports/www/trac.
mysql replication error
MySQL replication broke after a server crash. When trying to start it backup this error shows up.
100303 10:09:54 [ERROR] Slave: Query caused different errors on master and slave. Error on master: 'Invalid error code' (126), Error on slave: 'no error' (0). Default database: 'db_name'. Query: 'INSERT DELAYED INTO stats (clientID, date_accessed, current_url, referred_by, unique_id)
VALUES (0, now(), '/index.php', '', '076c47a32066f611d60e107b3c6be475')', Error_code: 0
100303 10:09:54 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'mysql-bin.000463' position 207592
After a little searching, in order to fix the problem I just have to repair the table with the following command.
mysql> repair table stats;
In my case the query is fine, the reason why it is failing may be
index file is crashed or file format is wrong. Here is a list of error codes that may be helpful to other people.
126 = Index file is crashed / Wrong file format
127 = Record-file is crashed
132 = Old database file
134 = Record was already deleted (or record file crashed)
135 = No more room in record file
136 = No more room in index file
141 = Duplicate unique key or constraint on write or update
144 = Table is crashed and last repair failed
145 = Table was marked as crashed and should be repaired
28 = You ran out of disk space. Delete unnecesary files, or move the MySQL data files to another partition.
1045: Access denied for user: = You forgot to specify a password, or you used the wrong one. Use the -u and -p options when starting the client. (use mysql --help to see how)
1046: No Database Selected = You must use the command USE database_name or the mysql_select_db() function to select a database to use.
1062: Duplicate entry 'x' for key n = You have attempted to insert or update data that would break the integrity of a UNIQUE index (most likely the primary key). You must redefine either the data you're trying to insert, or the indexes which you have imposed on the table.
1064: Syntax error = You wrote something that doesn't match the syntax definitions in the manual. When you have checked everything else, here's something to consider: A common error is to put a space between a function name and its opening parenthesis '('. Another common error is to use a reserved word out of its normal context.
2000: parse error near ... = see explanation for 1064: Syntax error
2002: Can't connect to local MySQL server = The MySQL socket file (usually, /tmp/mysql.sock) is missing. See the notes on connecting to the server.
vi search and replace ^M
To easily search and replace the ^M (new line) character in vi use the following command. Make sure your in command mode and not input mode when you do it.
:%s/^M$//g
to get the proper ^M in vi you'll need to do CTRL-v CTRL-M and that should display ^M
Installing Apache Tomcat 6 on FreeBSD
So you want to install Tomcat 6 on FreeBSD, this is how I did it.
cd /usr/ports/www/tomcat6
make install clean
use the default Options for diablo-jdk
Then you get some nice "licensing restrictions"
Because of licensing restrictions, you must fetch the distribution
manually.
Please access
http://www.FreeBSDFoundation.org/cgi-bin/download?download=diablo-caffe-freebsd6-i386-1.6.0_07-b02.tar.bz2
with a web browser and "Accept" the End User License Agreement for
"Caffe Diablo 1.6.0".
Please open http://java.sun.com/javase/downloads/index.jsp
in a web browser and follow the "Download" link for
"JDK US DST Timezone Update Tool - 1_3_21" to obtain the
time zone update file, tzupdater-1_3_21-2009p.zip.
Please place the downloaded file(s) in /usr/ports/distfiles.
*** Error code 1
Stop in /usr/ports/java/diablo-jdk16.
*** Error code 1Stop in /usr/ports/java/diablo-jdk16.
*** Error code 1Stop in /usr/ports/www/tomcat6.
So we go to the freebsd foundation website and download diablo caffe.
Then we go to the sun websute to download the Timezone update and what do you know, Sun has version 1.3.25 and not 1.3.21.
So we do an "cvsup -g -L 2 /root/cvs-supfile" on our system to update the ports collection.
... wait 10..20 minutes as the system wasn't up to date ...
While that's going, we copy (sftp) the file over to the server "diablo-caffe-freebsd6-i386-1.6.0_07-b02.tar.bz2"
cp /home/user_name/diablo-caffe-freebsd6-i386-1.6.0_07-b02.tar.bz2 /usr/ports/distfiles
The ports collection was updated but it didn't do anything, still need 1.3.21... Maybe because it's an old version of FreeBSD (6.2) o_0
Luckily for me, I had tzupdater-1_3_21-2009p.zip from a pervious install on another server (email me if you need the file)
I rsync the file accross
rsync -ave ssh /usr/ports/distfiles/tzupdater-1_3_21-2009p.zip us[email protected]:/home/user_name/tzupdater-1_3_21-2009p.zip
cp /home/user_name/tzupdater-1_3_21-2009p.zip /usr/ports/distfiles
cd /usr/ports/www/tomcat6
make install clean
The installation continues. Then it finishes without issue.
cd /usr/local/apache-tomcat-6.0
vi RUNNING.txt
To run it, we need to set up the JAVA_HOME environment variable... I'm on the bash shell (GO BASH) and we installed the JDK not JRE so I do
JAVA_HOME=/usr/local/diablo-jdk1.6.0
cd /usr/local/etc/rc.d/
./tomcat6 start
and nothing happened... Probably because it's not in /etc/rc.conf
cd /usr/local/apache-tomcat-6.0/logs/
ls -l
no logs in the log folder...
cd /usr/local/apache-tomcat-6.0/bin/
./startup.sh
Now we have logs
tail catalina.out
The following line shows up...
Feb 5, 2010 10:17:01 AM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-8180
so we go to http://192.168.1.101:8180 and just like that it comes up..
If you want to change the port, edit /usr/local/apache-tomcat-6.0/conf/server.xml
Now that was a long install, hope it helps someone.
postfix bcc on all emails
Track the email messages that make their way across your system.
What are those reasons? You may have noticed an increase in email abuse, either internal or external, Where forwarding a copy of each message to an administrator would be desirable.
If your looking just to get bcc'ed on a users email, please look at my previous post.
# cd /etc/postfix
# vi main.cf
//add this line to the bottom of your main.cf
[email protected]
save and quit then restart postfix.
# cd /usr/local/sbin
# ./postfix reload
now every email that gets sent through that machine, [email protected] will get bcc'ed on it.