寫了個備份mysql並scp遠程備份的腳本，大致過程如下
#!/bin/bash
list="database1 database2 database3 ... database n"
for database in $list
do
mysql -u root -ppasswd $database > $database.sql
scp $database.sql 192.168.1.2:
done
單獨執行沒有任何問題，放在crontab執行發現只有9個文件scp過去了，最後幾個全都收到了ssh認證失敗的錯誤，這個是怎麼回事？
ssh_exchange_identification: Connection closed by remote host^M
lost connection
database n 為啥多個空格
細節就不要在意了，只是描述一個大致的演演算法而已。實際中是用show databases;之後賦值給變數再for循環的
幾乎已經可以確定是crontab後台執行引起的問題了，./mysql_backup.sh & 這樣執行的話就收到了ssh_exchange的報錯。
難道後台執行有什麼玄機嗎？
看了下遠程伺服器的auth.log日誌，發現了這樣的問題。
直接執行./mysql_backup.sh，日誌中是這樣的：
May 18 23:25:26 ok988com sshd: pam_sm_authenticate: username =
May 18 23:25:26 ok988com sshd: Passphrase file wrapped
May 18 23:25:27 ok988com sshd: pam_unix(sshd:session): session opened for user ok988 by (uid=0)
May 18 23:25:27 ok988com sshd: pam_unix(sshd:session): session closed for user ok988
May 18 23:25:34 ok988com sshd: pam_sm_authenticate: Called
May 18 23:25:34 ok988com sshd: pam_sm_authenticate: username =
May 18 23:25:34 ok988com sshd: Passphrase file wrapped
May 18 23:25:35 ok988com sshd: pam_unix(sshd:session): session opened for user ok988 by (uid=0)
May 18 23:25:35 ok988com sshd: pam_unix(sshd:session): session closed for user ok988
May 18 23:25:41 ok988com sshd: pam_sm_authenticate: Called
May 18 23:25:41 ok988com sshd: pam_sm_authenticate: username =
May 18 23:25:41 ok988com sshd: Passphrase file wrapped
May 18 23:25:42 ok988com sshd: pam_unix(sshd:session): session opened for user ok988 by (uid=0)
May 18 23:25:42 ok988com sshd: pam_unix(sshd:session): session closed for user ok988
May 18 23:25:48 ok988com sshd: pam_sm_authenticate: Called
May 18 23:25:48 ok988com sshd: pam_sm_authenticate: username =
May 18 23:25:48 ok988com sshd: Passphrase file wrapped
May 18 23:25:49 ok988com sshd: pam_unix(sshd:session): session opened for user ok988 by (uid=0)
May 18 23:25:49 ok988com sshd: pam_unix(sshd:session): session closed for user ok988
May 18 23:25:54 ok988com sshd: pam_sm_authenticate: Called
May 18 23:25:54 ok988com sshd: pam_sm_authenticate: username =
……
以下省略。
可以看到sshd的進程很有規律性，scp一個文件的時候打開ssh通道，傳輸完畢之後關閉這個通道，然後繼續打開下一個ssh通道，scp傳輸下一個文件
crontab執行這個腳本的時候，auth.log是這樣的。
May 17 04:00:08 ok988com sshd: pam_sm_authenticate: Called
May 17 04:00:08 ok988com sshd: pam_sm_authenticate: username =
May 17 04:00:08 ok988com sshd: Passphrase file wrapped
May 17 04:00:09 ok988com sshd: Accepted password for ok988 from 擦掉IP port 48714 ssh2
May 17 04:00:09 ok988com sshd: pam_unix(sshd:session): session opened for user ok988 by (uid=0)
May 17 04:00:09 ok988com sshd: Received disconnect from 擦掉IP: 11: disconnected by user
May 17 04:00:09 ok988com sshd: pam_unix(sshd:session): session closed for user ok988
May 17 04:00:15 ok988com sshd: pam_sm_authenticate: Called
May 17 04:00:15 ok988com sshd: pam_sm_authenticate: username =
May 17 04:00:15 ok988com sshd: Passphrase file wrapped
May 17 04:00:16 ok988com sshd: Accepted password for ok988 from 擦掉IP port 48718 ssh2
May 17 04:00:16 ok988com sshd: pam_unix(sshd:session): session opened for user ok988 by (uid=0)
May 17 04:00:16 ok988com sshd: Received disconnect from 擦掉IP: 11: disconnected by user
May 17 04:00:16 ok988com sshd: pam_unix(sshd:session): session closed for user ok988
May 17 04:00:24 ok988com sshd: pam_sm_authenticate: Called
May 17 04:00:24 ok988com sshd: pam_sm_authenticate: username =
May 17 04:00:24 ok988com sshd: Passphrase file wrapped
May 17 04:00:25 ok988com sshd: Accepted password for ok988 from 擦掉IP port 48720 ssh2
May 17 04:00:25 ok988com sshd: pam_unix(sshd:session): session opened for user ok988 by (uid=0)
May 17 04:00:25 ok988com sshd: Received disconnect from 擦掉IP: 11: disconnected by user
May 17 04:00:25 ok988com sshd: pam_unix(sshd:session): session closed for user ok988
May 17 04:00:31 ok988com sshd: pam_sm_authenticate: Called
May 17 04:00:31 ok988com sshd: pam_sm_authenticate: username =
May 17 04:00:31 ok988com sshd: Passphrase file wrapped
May 17 04:00:32 ok988com sshd: Accepted password for ok988 from 擦掉IP port 48726 ssh2
May 17 04:00:32 ok988com sshd: pam_unix(sshd:session): session opened for user ok988 by (uid=0)
May 17 04:00:32 ok988com sshd: Received disconnect from 擦掉IP: 11: disconnected by user
May 17 04:00:32 ok988com sshd: pam_unix(sshd:session): session closed for user ok988
May 17 04:00:38 ok988com sshd: pam_sm_authenticate: Called
May 17 04:00:38 ok988com sshd: pam_sm_authenticate: username =
May 17 04:00:38 ok988com sshd: Passphrase file wrapped
May 17 04:00:39 ok988com sshd: Accepted password for ok988 from 擦掉IP port 48727 ssh2
May 17 04:00:39 ok988com sshd: pam_unix(sshd:session): session opened for user ok988 by (uid=0)
May 17 04:00:39 ok988com sshd: Received disconnect from 擦掉IP: 11: disconnected by user
May 17 04:00:39 ok988com sshd: pam_unix(sshd:session): session closed for user ok988
May 17 04:00:44 ok988com sshd: pam_sm_authenticate: Called
May 17 04:00:44 ok988com sshd: pam_sm_authenticate: username =
May 17 04:00:44 ok988com sshd: Passphrase file wrapped
依然很有規律，但是確實只有9個sshd。後來不知道怎麼就想到了多線程的問題，shell中實現多線程的方式是後台執行，隱隱約約記得好像crontab就是後台執行的，再一翻man手冊,man sshd_config，發現有一個MaxStartup參數，定義了最大允許的遠程ssh的數量，默認是10，剛好9個文件傳輸通道加上1個ssh登錄認證通道，這不就剛好是10個么？
然後果斷選擇後台執行腳本./mysql_backup.sh，果然重現了這個問題，看來後台執行真的跟前台執行有很大的不同，後台執行的時候scp並沒有把自己的ssh通道關閉（但是為什麼會看到Received disconnect from 擦掉IP: 11: disconnected by user？百思不得其解），導致ssh通道佔滿了默認的10個，後面就再也沒有ssh認證信息了，全部被拒絕了。
client close connect 是 Tcpip 結束會話的第一步。
你可以用scp -v 看一下。並使用tcpdump 抓包看看，前台和後台的區別。
注在後台運行hou，手動ssh 登陸scp dest server 能上去么？
後台是可以上去的，不然也不會成功傳輸9個文件了。
發現後台跟前台運行在目標伺服器上的auth.log顯示是不一樣的，還是不太明白為什麼後台執行的時候對方的auth.log明明顯示到了收到disconnect的消息，但是卻沒有真正斷開SSH通道？
TCP/IP斷開連接是需要4次揮手。你看到client段發送斷開連接請求，但是如果伺服器端沒有回應說我也發送完數據。那麼這個連接是沒有斷開的。
你netstat -talntp|grep 22 輸出看下,伺服器端的狀態.
這次真是奇怪了，用for循環netstat -anltp，發現這次的SSH埠是一次開放一個，斷掉之後繼續下一次循環，繼續開啟ssh埠，和前端執行完全一樣，並沒有重現那個問題
算了不折騰了，把scp放到for循環之外吧，每次循環的時候把文件名加入到文件列表中，執行一次scp就行了，這樣就沒有後顧之憂了。
