注意： Apache Archiva 2024-02 开始已经停止维护 建议使用 JFrog Artifactory 替代。
最近使用自己搭建的Apache Archiva来代理Maven仓库，经常发生失败的情况，观察Archiva的日志(logs/archiva.log)，看到如下的内容：
2016-11-22 19:52:02,773 [ajp-bio-127.0.0.1-8009-exec-74] WARN org.apache.archiva.proxy.DefaultRepositoryProxyConnectors [] - Transfer error from repository central for artifact org.mockito:mockito-core:2.2.22::jar , continuing to next repository. Error message: Download failure on resource [https://repo.maven.apache.org/maven2/org/mockito/mockito-core/2.2.22/mockito-core-2.2.22.jar]:GET request of: org/mockito/mockito-core/2.2.22/mockito-core-2.2.22.jar from central failed (cause: java.net.SocketTimeoutException: Read timed out)
2016-11-22 19:52:02,773 [ajp-bio-127.0.0.1-8009-exec-74] ERROR org.apache.archiva.webdav.ArchivaDavResourceFactory [] - Failures occurred downloading from some remote repositories:
central: Download failure on resource [https://repo.maven.apache.org/maven2/org/mockito/mockito-core/2.2.22/mockito-core-2.2.22.jar]:GET request of: org/mockito/mockito-core/2.2.22/mockito-core-2.2.22.jar from central failed (cause: java.net.SocketTimeoutException: Read timed out)
org.apache.archiva.policies.ProxyDownloadException: Failures occurred downloading from some remote repositories:
central: Download failure on resource [https://repo.maven.apache.org/maven2/org/mockito/mockito-core/2.2.22/mockito-core-2.2.22.jar]:GET request of: org/mockito/mockito-core/2.2.22/mockito-core-2.2.22.jar from central failed (cause: java.net.SocketTimeoutException: Read timed out)
at org.apache.archiva.proxy.DefaultRepositoryProxyConnectors.fetchFromProxies(DefaultRepositoryProxyConnectors.java:366) ~[archiva-proxy-2.2.1.jar:?]
at org.apache.archiva.webdav.ArchivaDavResourceFactory.fetchContentFromProxies(ArchivaDavResourceFactory.java:820) [archiva-webdav-2.2.1.jar:?]
at org.apache.archiva.webdav.ArchivaDavResourceFactory.processRepository(ArchivaDavResourceFactory.java:629) [archiva-webdav-2.2.1.jar:?]
at org.apache.archiva.webdav.ArchivaDavResourceFactory.createResource(ArchivaDavResourceFactory.java:325) [archiva-webdav-2.2.1.jar:?]
at org.apache.archiva.webdav.RepositoryServlet.service(RepositoryServlet.java:126) [archiva-webdav-2.2.1.jar:?]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) [servlet-api-3.0.jar:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) [tomcat-catalina-7.0.52.jar:7.0.52]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [tomcat-catalina-7.0.52.jar:7.0.52]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) [tomcat-catalina-7.0.52.jar:7.0.52]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) [tomcat-catalina-7.0.52.jar:7.0.52]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501) [tomcat-catalina-7.0.52.jar:7.0.52]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) [tomcat-catalina-7.0.52.jar:7.0.52]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98) [tomcat-catalina-7.0.52.jar:7.0.52]
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950) [tomcat-catalina-7.0.52.jar:7.0.52]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) [tomcat-catalina-7.0.52.jar:7.0.52]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) [tomcat-catalina-7.0.52.jar:7.0.52]
at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:193) [tomcat-coyote-7.0.52.jar:7.0.52]
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607) [tomcat-coyote-7.0.52.jar:7.0.52]
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313) [tomcat-coyote-7.0.52.jar:7.0.52]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_111]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_111]
at java.lang.Thread.run(Thread.java:745) [?:1.7.0_111]
明显是从https://repo.maven.apache.org/maven2这个地址下载数据的时候发生了异常。这个仓库的地址是Apache Archiva中的默认仓库地址。从目前的测试来看，这个地址在国内访问，经常出现问题。对于国内用户来说https://repo1.maven.org/maven2这个中央仓库的地址是相对来说更加稳定。因此只要在Remote Repositories中增加这个中央仓库地址即可。
如下图操作：
另外，在添加完成后，顺便在属性中修改一下Download Timeout，从默认的60秒修改到600秒，减少超时的发生即可。
如上操作只能部分解决问题，在现实过程中，依旧会发生失败，失败主要集中在下载https://repo1.maven.org/maven2/.index/nexus-maven-repository-index.gz这个索引文件的时候，这个索引文件有300-400MB的样子，一次完整的下载基本上是都会失败，要命的是Apache Archiva在处理这个文件的时候，基本上没有进行任何容错处理。这个时候我们要么修改源代码来修正，要么需要辅助Apache Archiva完成这个文件的下载。
下面，我们实验通过Linux定时任务，nginx，aria2来实现对Apache Archiva下载的辅助处理。
1.首先安装必须的软件
$ sudo apt-get install nginx $ sudo apt-get install aria2
2.接下来，配置nginx
$ sudo vim /etc/nginx/sites-enabled/default
整个配置文件的原文如下：
##
# You should look at the following URL's in order to grasp a solid understanding
# of Nginx configuration files in order to fully unleash the power of Nginx.
# http://wiki.nginx.org/Pitfalls
# http://wiki.nginx.org/QuickStart
# http://wiki.nginx.org/Configuration
#
# Generally, you will want to move this file somewhere, and start with a clean
# file but keep this around for reference. Or just disable in sites-enabled.
#
# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
##
# Default server configuration
#
server {
listen 80 default_server;
listen [::]:80 default_server;
# SSL configuration
#
# listen 443 ssl default_server;
# listen [::]:443 ssl default_server;
#
# Note: You should disable gzip for SSL traffic.
# See: https://bugs.debian.org/773332
#
# Read up on ssl_ciphers to ensure a secure configuration.
# See: https://bugs.debian.org/765782
#
# Self signed certs generated by the ssl-cert package
# Don't use them in a production server!
#
# include snippets/snakeoil.conf;
root /var/www/html;
# Add index.php to the list if you are using PHP
index index.html index.htm index.nginx-debian.html;
server_name _;
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files $uri $uri/ =404;
}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
#location ~ \.php$ {
# include snippets/fastcgi-php.conf;
#
# # With php7.0-cgi alone:
# fastcgi_pass 127.0.0.1:9000;
# # With php7.0-fpm:
# fastcgi_pass unix:/run/php/php7.0-fpm.sock;
#}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /\.ht {
# deny all;
#}
}
# Virtual Host configuration for example.com
#
# You can move that to a different file under sites-available/ and symlink that
# to sites-enabled/ to enable it.
#
#server {
# listen 80;
# listen [::]:80;
#
# server_name example.com;
#
# root /var/www/example.com;
# index index.html;
#
# location / {
# try_files $uri $uri/ =404;
# }
#}
修改后的结果如下：
##
# You should look at the following URL's in order to grasp a solid understanding
# of Nginx configuration files in order to fully unleash the power of Nginx.
# http://wiki.nginx.org/Pitfalls
# http://wiki.nginx.org/QuickStart
# http://wiki.nginx.org/Configuration
#
# Generally, you will want to move this file somewhere, and start with a clean
# file but keep this around for reference. Or just disable in sites-enabled.
#
# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
##
# Default server configuration
#
server {
#listen 80 default_server;
#listen [::]:80 default_server;
listen 127.0.0.1:8090 default_server;
# SSL configuration
#
# listen 443 ssl default_server;
# listen [::]:443 ssl default_server;
#
# Note: You should disable gzip for SSL traffic.
# See: https://bugs.debian.org/773332
#
# Read up on ssl_ciphers to ensure a secure configuration.
# See: https://bugs.debian.org/765782
#
# Self signed certs generated by the ssl-cert package
# Don't use them in a production server!
#
# include snippets/snakeoil.conf;
root /data/nginx/maven_index;
# Add index.php to the list if you are using PHP
index index.html index.htm index.nginx-debian.html;
server_name _;
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files $uri $uri/ =404;
resolver 114.114.114.114 218.85.152.99;
resolver_timeout 30s;
#nginx 不支持if嵌套也不支持多条件判断，因此只能用下面的方式来模拟
#留意判断语句与括号之间的空格，缺失空格会导致语法错误
#if ( ( $host ~* "repo1\.maven\.org" ) && ( $request_uri ~* "maven2/\.index" ) ) {
set $flag 0;
if ( $host ~* "repo1\.maven\.org" ) {
set $flag "${flag}1";
}
if ( $request_uri ~* "maven2/\.index" ) {
set $flag "${flag}2";
}
if ($flag = "012") {
proxy_pass http://127.0.0.1:8090$request_uri;
}
#避免循环重定向问题
if ( $host != "127.0.0.1" ) {
proxy_pass http://$host$request_uri;
}
}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
#location ~ \.php$ {
# include snippets/fastcgi-php.conf;
#
# # With php7.0-cgi alone:
# fastcgi_pass 127.0.0.1:9000;
# # With php7.0-fpm:
# fastcgi_pass unix:/run/php/php7.0-fpm.sock;
#}
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
#location ~ /\.ht {
# deny all;
#}
}
# Virtual Host configuration for example.com
#
# You can move that to a different file under sites-available/ and symlink that
# to sites-enabled/ to enable it.
#
#server {
# listen 80;
# listen [::]:80;
#
# server_name example.com;
#
# root /var/www/example.com;
# index index.html;
#
# location / {
# try_files $uri $uri/ =404;
# }
#}
接下来，重启nginx服务。
3.设置定时任务，定时检查远端服务器上的数据是否有更新
任务脚本内容如下：
nginx_dir="/data/nginx"
mvn_idx_dir="$nginx_dir/maven2/.index"
mvn_idx_dl_dir="$nginx_dir/maven2/.CacheIndex"
mvn_idx_name="nexus-maven-repository-index"
mvn_idx_path_name="$mvn_idx_dir/$mvn_idx_name"
mvn_idx_gz_name="$mvn_idx_dir/$mvn_idx_name.gz"
mvn_idx_prop_name="$mvn_idx_dir/$mvn_idx_name.properties"
mvn_idx_dl_gz_name="$mvn_idx_dl_dir/$mvn_idx_name.gz"
mvn_idx_dl_prop_name="$mvn_idx_dl_dir/$mvn_idx_name.properties"
log_file="/data/nginx/maven_index_log.txt"
dt_fmt="`date "+%Y-%m-%d %H:%M:%S"`"
if [ ! -d $mvn_idx_dl_dir ]; then
mkdir -p $mvn_idx_dl_dir
fi
if [ ! -d $mvn_idx_dir ]; then
mkdir -p $mvn_idx_dir
fi
if [ ! -d $mvn_idx_dl_dir ]; then
echo "$dt_fmt mkdir $mvn_idx_dl_dir failed !" >> $log_file
exit -1
fi
if [ ! -d $mvn_idx_dir ]; then
echo "$dt_fmt mkdir $mvn_idx_dir failed !" >> $log_file
exit -1
fi
cd $mvn_idx_dl_dir
dl_idx="true"
dl_prop="true"
#echo mvn_idx_gz_name="$mvn_idx_gz_name"
#先检查是不是需要下载文件，然后再下载文件，减少不必要的网络请求数据
if [ -f "$mvn_idx_gz_name" ]; then
remote_idx_md5=$(curl -s https://repo1.maven.org/maven2/.index/nexus-maven-repository-index.gz.md5)
if [ "${#remote_idx_md5}" != "32" ] ; then
echo "$dt_fmt download $mvn_idx_gz_name.md5 failed !" >> $log_file
exit -1
fi
local_idx_md5=$(md5sum $mvn_idx_gz_name | cut -b 1-32)
#echo remote_idx_md5="$remote_idx_md5" local_idx_md5="$local_idx_md5"
if [ "$remote_idx_md5" = "$local_idx_md5" ] ; then
dl_idx="false"
if [ -f "$mvn_idx_prop_name" ]; then
remote_prop_md5=$(curl -s https://repo1.maven.org/maven2/.index/nexus-maven-repository-index.properties.md5)
if [ "${#remote_prop_md5}" != "32" ] ; then
echo "$dt_fmt download $mvn_idx_prop_name failed !" >> $log_file
exit -1
fi
local_prop_md5=$(md5sum $mvn_idx_prop_name | cut -b 1-32)
if [ "$remote_prop_md5" = "$local_prop_md5" ] ; then
dl_prop="false"
echo "$dt_fmt check file success ,no need to update !" >> $log_file
exit 0
fi
fi
fi
fi
if [ "$dl_idx" = "true" ] ; then
aria2c -c https://repo1.maven.org/maven2/.index/nexus-maven-repository-index.gz
aria2c -c https://repo1.maven.org/maven2/.index/nexus-maven-repository-index.gz.md5
aria2c -c https://repo1.maven.org/maven2/.index/nexus-maven-repository-index.gz.sha1
fi
if [ "$dl_prop" = "true" ] ; then
aria2c -c https://repo1.maven.org/maven2/.index/nexus-maven-repository-index.properties
aria2c -c https://repo1.maven.org/maven2/.index/nexus-maven-repository-index.properties.md5
aria2c -c https://repo1.maven.org/maven2/.index/nexus-maven-repository-index.properties.sha1
fi
#校验下载到的文件
if [ "$dl_idx" = "true" ] ; then
dl_idx_md5=$(md5sum $mvn_idx_dl_gz_name | cut -b 1-32)
dl_idx_f_md5=$(cat $mvn_idx_dl_gz_name.md5)
if [ "$dl_idx_md5" = "$dl_idx_f_md5" ] ; then
rm -rf $mvn_idx_gz_name
mv $mvn_idx_dl_gz_name $mvn_idx_gz_name
rm -rf $mvn_idx_gz_name.md5
mv $mvn_idx_dl_gz_name.md5 $mvn_idx_gz_name.md5
rm -rf $mvn_idx_gz_name.sha1
mv $mvn_idx_dl_gz_name.sha1 $mvn_idx_gz_name.sha1
else
echo "$dt_fmt check downloaded index file failed !" >> $log_file
cd $nginx_dir
rm -rf $mvn_idx_dl_dir
fi
fi
if [ "$dl_prop" = "true" ] ; then
dl_prop_md5=$(md5sum $mvn_idx_dl_prop_name | cut -b 1-32)
dl_prop_f_md5=$(cat $mvn_idx_dl_prop_name.md5)
if [ "$dl_prop_md5" = "$dl_prop_f_md5" ] ; then
rm -rf $mvn_idx_prop_name
mv $mvn_idx_dl_prop_name $mvn_idx_prop_name
rm -rf $mvn_idx_prop_name.md5
mv $mvn_idx_dl_prop_name.md5 $mvn_idx_prop_name.md5
rm -rf $mvn_idx_prop_name.sha1
mv $mvn_idx_dl_prop_name.sha1 $mvn_idx_prop_name.sha1
else
echo "$dt_fmt check downloaded properties file failed !" >> $log_file
cd $nginx_dir
rm -rf $mvn_idx_dl_dir
fi
fi
默认我们把脚本执行路径为/data/nginx/mvn_index_corn.sh。
设置定时任务的脚本如下：
chmod +x /data/nginx/mvn_index_corn.sh #write out current crontab crontab -l > addcron #echo new cron into cron file ,每隔30分钟我们调度一次任务,前面是文件锁，防止并发冲突 echo "30 * * * * flock -x -w 10 /dev/shm/mvn_index_corn.lock -c \"sh /data/nginx/mvn_index_corn.sh\"" >> addcron #install new cron file crontab addcron rm addcron
执行上面的脚本。
4.设置Apache Archiva的代理服务器配置
