Jenkins阻止Execute shell输出命令内容

默认情况下,Jenkins在构建的时候会输出脚本中命令的具体内容,原因是在执行脚本的时候,使用的是`-e`参数执行的。

但是当我们的脚本中存在用户名,密码等关键信息的时候,这些内容会被输出到编译日志中,这样就容易导致安全问题。

解决办法是在脚本的最前面增加`#!/bin/bash`,来覆盖脚本执行的默认参数,如下图:

继续阅读Jenkins阻止Execute shell输出命令内容

Centos 7配置Jenkins构建Android持续集成(离线内网环境)

安装配置Jenkins

# 要求Java 1.8
$ yum install java

$ yum install subversion

$ yum install tomcat

$ /bin/systemctl restart tomcat.service

$ wget https://get.jenkins.io/war/2.254/jenkins.war

$ cp jenkins.war /usr/share/tomcat/webapps/

$ /bin/systemctl restart tomcat.service

# 服务器访问地址
# http://158.220.155.188:8080/jenkins/

# apache
$ yum install httpd

$ /bin/systemctl restart httpd.service

# apache 默认工作目录 /var/www/html/

为了安全考虑,首先需要解锁Jenkins

继续阅读Centos 7配置Jenkins构建Android持续集成(离线内网环境)

Gradle传递System Property

最近在搭建`Jenkins`环境,实现`Android`自动化编译的过程中,由于内网服务器不能访问外网,因此只能配置`Robolectric`访问内网的服务器。

根据官方文档 Configuring Robolectric,发现如果要更改`Robolectric`默认的下载服务器链接地址,需要在项目的每个`lib`库中都配置如下参数,才能实现:

android {
  testOptions {
    unitTests.all {
      systemProperty 'robolectric.dependency.repo.url', 'https://local-mirror/repo'
      systemProperty 'robolectric.dependency.repo.id', 'local'
    }
  }
}

但是这样配置有一个问题,就是没办法动态调整链接地址。

而我们使用

$ bash gradlew clean build -Drobolectric.dependency.repo.url=http://127.0.0.1/jcenter

手工指定的参数,并没有在编译的时候生效。

参数丢失的原因是因为测试用例在新的`JVM`中执行,传入的参数不会自动带给新创建的`JVM`。

这时需要在`Gradle`脚本中将读到的值重新设到系统属性里面,才可以被`Java`程序读到。

android {
    testOptions {
        unitTests.all {
            //命令行下 单元测试可能卡住的问题
            jvmArgs '-noverify'
            //robolectric外部指定下载资源链接的参数,使用 -D 参数指定 bash gradlew clean build -Drobolectric.dependency.repo.url=http://127.0.0.1/jcenter
            systemProperty 'robolectric.dependency.repo.url', System.getProperty("robolectric.dependency.repo.url")
            systemProperty 'robolectric.dependency.repo.id', System.getProperty("robolectric.dependency.repo.id")
        }
    }
}

每个项目的`build.gradle`中都增加上面的配置之后,就可以保证在外部编译的时候动态指定参数了。

Android Studio Chipmunk | 2021.2.1 Patch 1 以及以上的版本,可能会发生如下报错:

URI is not absolute
java.lang.IllegalArgumentException: URI is not absolute
	at java.base/java.net.URL.fromURI(URL.java:692)
	at java.base/java.net.URI.toURL(URI.java:1116)
	at org.robolectric.internal.dependency.MavenArtifactFetcher.getRemoteUrl(MavenArtifactFetcher.java:138)
	at org.robolectric.internal.dependency.MavenArtifactFetcher.fetchToStagingRepository(MavenArtifactFetcher.java:145)
	at org.robolectric.internal.dependency.MavenArtifactFetcher.fetchArtifact(MavenArtifactFetcher.java:65)
	at org.robolectric.internal.dependency.MavenDependencyResolver.lambda$getLocalArtifactUrls$0(MavenDependencyResolver.java:80)
	at org.robolectric.internal.dependency.MavenDependencyResolver.whileLocked(MavenDependencyResolver.java:100)
	at org.robolectric.internal.dependency.MavenDependencyResolver.getLocalArtifactUrls(MavenDependencyResolver.java:75)
	at org.robolectric.internal.dependency.MavenDependencyResolver.getLocalArtifactUrls(MavenDependencyResolver.java:65)
	at org.robolectric.internal.dependency.MavenDependencyResolver.getLocalArtifactUrl(MavenDependencyResolver.java:116)
	at org.robolectric.plugins.LegacyDependencyResolver.getLocalArtifactUrl(LegacyDependencyResolver.java:89)
	at org.robolectric.plugins.DefaultSdkProvider$DefaultSdk.getJarPath(DefaultSdkProvider.java:146)
	at org.robolectric.internal.AndroidSandbox$SdkSandboxClassLoader.<init>(AndroidSandbox.java:102)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at org.robolectric.util.inject.Injector.inject(Injector.java:250)
	at org.robolectric.util.inject.Injector.lambda$memoized$1(Injector.java:232)
	at org.robolectric.util.inject.Injector$MemoizingProvider.get(Injector.java:498)
	at org.robolectric.util.inject.Injector.getInstanceInternal(Injector.java:224)
	at org.robolectric.util.inject.Injector.resolveDependencies(Injector.java:296)
	at org.robolectric.util.inject.Injector.inject(Injector.java:248)
	at org.robolectric.util.inject.Injector.lambda$memoized$1(Injector.java:232)
	at org.robolectric.util.inject.Injector$MemoizingProvider.get(Injector.java:498)
	at org.robolectric.util.inject.Injector.getInstanceInternal(Injector.java:224)
	at org.robolectric.util.inject.Injector.getInstance(Injector.java:208)
	at org.robolectric.util.inject.Injector.access$700(Injector.java:96)
	at org.robolectric.util.inject.Injector$ScopeBuilderProvider.create(Injector.java:564)
	at org.robolectric.util.inject.Injector$ScopeBuilderProvider.lambda$get$0(Injector.java:547)
	at com.sun.proxy.$Proxy19.build(Unknown Source)
	at org.robolectric.internal.SandboxManager.getAndroidSandbox(SandboxManager.java:57)
	at org.robolectric.RobolectricTestRunner.getSandbox(RobolectricTestRunner.java:285)
	at org.robolectric.RobolectricTestRunner.getSandbox(RobolectricTestRunner.java:68)
	at org.robolectric.internal.SandboxTestRunner$2.evaluate(SandboxTestRunner.java:236)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.robolectric.internal.SandboxTestRunner$1.evaluate(SandboxTestRunner.java:93)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)

该报错的原因是 Android Studio 没有正确读取我们配置的 robolectric.dependency.repo.url 环境变量,导致无法成功下载,当然也可能是由于高版本不支持 HTTP ,链接地址必须是 HTTPS 导致此问题,这个没有深究。

解决方法是,在命令行执行

$ bash gradlew clean build -Drobolectric.dependency.repo.url=http://127.0.0.1/jcenter

确保在当前用户目录下的 .m2 目录下,成功完成 robolectric 需要的全部文件的下载。这样,执行测试用例的时候,就可以从本地获取依赖文件,而不需要再去服务器上获取。

参考链接


Ubuntu Server 18.04 LTS隐藏Tomcat-9.0.16.0的版本号与操作系统类型

推荐方案

另外更推荐的方法是通过 Tomcat 的配置文件完成,而不是修改代码,具体配置方法为:

conf/server.xml 配置文件中的 <Host> 配置项中添加如下配置:

<Valve className="org.apache.catalina.valves.ErrorReportValve" showReport="false" showServerInfo="false" />

配置项说明:

  • showReport:默认值为true,默认显示报错信息
  • showServerInfo:默认值为true,默认显示Tomcat的版本号

其他方案

其他的方案跟  Ubuntu 14.04隐藏Tomcat-7.0.52的版本号与操作系统类型 是一致的,但是具体的细节上存在不小的差异,还是需要记录一下。

$ cd ~

$ mkdir catalina

$ cd catalina

$ cp /usr/share/tomcat9/lib/catalina.jar .

$ unzip catalina.jar

$ cd org/apache/catalina/util

$ vim ServerInfo.properties

可以看到里面的内容如下:

# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

server.info=Apache Tomcat/9.0.16 (Ubuntu)
server.number=9.0.16.0
server.built=Sep 11 2019 19:47:51 UTC

直接注释掉里面的内容,如下:

# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# server.info=Apache Tomcat/9.0.16 (Ubuntu)
# server.number=9.0.16.0
# server.built=Sep 11 2019 19:47:51 UTC

修改完成后,把修改完成的数据存储到catalina.jar中。

$ cd ~

$ cd catalina

$ sudo apt install openjdk-11-jdk-headless

$ jar uvf catalina.jar org/apache/catalina/util/ServerInfo.properties

把修改后的catalina.jar放回到Tomcat的目录下面:

$ cd ~

$ cd catalina

$ sudo unlink /usr/share/tomcat9/lib/catalina.jar 

$ sudo mv /usr/share/java/tomcat9-catalina.jar /usr/share/java/tomcat9-catalina.jar.old

$ sudo cp catalina.jar /usr/share/java/

$ sudo chmod +r /usr/share/java/catalina.jar

$ cd /usr/share/tomcat9/lib

$ sudo ln -s ../../java/catalina.jar catalina.jar

重启Tomcat的服务

$ sudo service tomcat9 restart

参考链接


腾讯云Ubuntu Server 16.04.7 LTS升级系统到Ubuntu Server 18.04.5 LTS之后letsencrypt证书更新出现异常“ImportError: cannot import name _remove_dead_weakref”

腾讯云`Ubuntu Server 16.04.7 LTS`升级系统到`Ubuntu Server 18.04.5 LTS`之后,`letsencrypt`证书更新出现异常,如下:

$ sudo ./letsencrypt-auto
Error: couldn't get currently installed version for /opt/eff.org/certbot/venv/bin/letsencrypt: 
Traceback (most recent call last):
  File "/opt/eff.org/certbot/venv/bin/letsencrypt", line 7, in <module>
    from certbot.main import main
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/main.py", line 2, in <module>
    from certbot._internal import main as internal_main
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/_internal/main.py", line 6, in <module>
    import logging.handlers
  File "/usr/lib/python2.7/logging/__init__.py", line 26, in <module>
    import sys, os, time, cStringIO, traceback, warnings, weakref, collections
  File "/usr/lib/python2.7/weakref.py", line 14, in <module>
    from _weakref import (
ImportError: cannot import name _remove_dead_weakref

原因为系统版本变化过大导致以前安装的`Python`组件不能适应最新的系统,最简单的方法就是删除之前安装的`Python`组件,让`letsencrypt`重新安装即可。

$ sudo apt-get install python-pip

$ sudo rm -rf /opt/eff.org/

$ sudo ./letsencrypt-auto

参考链接


Cannot renew certificate “ImportError: cannot import name _remove_dead_weakref”

腾讯云Ubuntu Server 16.04.7 LTS升级系统到Ubuntu Server 18.04.5 LTS

参考 阿里云ECS ubuntu 14.04.5 LTS升级到ubuntu 16.04.2 LTS / 阿里云从ubuntu16.04.5升级到ubuntu 18.04.3后调整PHP 7.0到PHP 7.2 整个的升级步骤是差不多的,腾讯的升级流程更加顺利。

但是在升级完成之后,遇到了一个域名解析问题,报错如下:

$ ping www.baidu.com
ping: www.baidu.com: Temporary failure in name resolution

$ sudo apt-get update
Err:1 http://mirrors.tencentyun.com/ubuntu bionic InRelease
  Temporary failure resolving 'mirrors.tencentyun.com'
Err:2 http://mirrors.tencentyun.com/ubuntu bionic-security InRelease
  Temporary failure resolving 'mirrors.tencentyun.com'
Err:3 http://mirrors.tencentyun.com/ubuntu bionic-updates InRelease
  Temporary failure resolving 'mirrors.tencentyun.com'

这个原因是因为我们在安装过程中,覆盖了腾讯设置的域名解析服务配置文件。

这个配置文件是`/etc/resolv.conf`。

但是,在`Ubuntu Server 18.04.5 LTS`系统上,直接修改`/etc/resolv.conf`的话,重启系统之后,设置会被还原。我们需要修改`/etc/systemd/resolved.conf`才行。

根据地域的不同,服务器的地址存在差异,参考下面的脚本即可。

广州服务器:

# 不存在则增加
$ sudo sed -i "s/^#DNS=.*/DNS=10.138.224.65 10.182.20.26 10.182.24.12/g" /etc/systemd/resolved.conf

# 存在则替换
$ sudo sed -i "s/^DNS=.*/DNS=10.138.224.65 10.182.20.26 10.182.24.12/g" /etc/systemd/resolved.conf

# 重启服务,配置生效
$ sudo systemctl restart systemd-resolved.service

上海服务器:

# 不存在则增加
$ sudo sed -i "s/^#DNS=.*/DNS=10.236.158.106 10.237.148.54 10.237.148.60/g" /etc/systemd/resolved.conf

# 存在则替换
$ sudo sed -i "s/^DNS=.*/DNS=10.236.158.106 10.237.148.54 10.237.148.60/g" /etc/systemd/resolved.conf

# 重启服务,配置生效
$ sudo systemctl restart systemd-resolved.service

香港服务器:

# 不存在则增加
$ sudo sed -i "s/^#DNS=.*/DNS=10.243.28.52 10.145.0.57 10.145.0.58/g" /etc/systemd/resolved.conf

# 存在则替换
$ sudo sed -i "s/^DNS=.*/DNS=10.243.28.52 10.145.0.57 10.145.0.58/g" /etc/systemd/resolved.conf

# 重启服务,配置生效
$ sudo systemctl restart systemd-resolved.service

参考链接