6 posts tagged with "kotlin"

View All Tags

使用apache-httpclient5并通过DNS请求服务如果域名不符合FQDN规范导致报错NullPointerException的问题的排查及修复

March 15, 2024 · 3 min read

orange

programmer on jvm platform

此问题是升级apache-httpclient5过程中遇到的问题.

项目是多租户场景, 每个租户都有自己的服务, 所有服务部署在kubernetes上.
每个租户的服务在独立的namespace中 namespace是租户的ID(例如1663783236729442304)

解决在Kotlin Coroutines中的命令行调用中出现Stream Closed异常

April 5, 2023 · 3 min read

orange

programmer on jvm platform

之前有一个服务内部需要调用外部程序(rclone), 于是我写了一个类来封装命令行调用, 该类主要是基于kotlinx.coroutines 来实现的.
代码如下:

CommandExecutorImpl.kt
import java.io.IOException
import java.io.InputStream

class CommandExecutorImpl : CommandExecutor, LogCapability {

	override suspend fun execute(options: CommandExecutionOptions) =
		coroutineScope {
			val command: String = options.command.joinToString(separator = " ")
			logger.info("$ {}", command)
			val process: Process = createProcess(options)

			val asyncReadStdOut = asyncRead(input = process.inputStream, consume = options.onNewStdoutRead)
			val asyncReadStderr = asyncRead(input = process.errorStream, consume = options.onNewStderrRead)
			try {
				while (process.isAlive) {
					delay(500)
				}
				if (process.exitValue() != 0) {
					throw IllegalStateException("Process exited with non-zero exit code")
				}
			} finally {
				// https://kotlinlang.org/docs/cancellation-and-timeouts.html#run-non-cancellable-block
				withContext(NonCancellable) {
					process.destroy()
					asyncReadStdOut.cancelAndJoin()
					asyncReadStderr.cancelAndJoin()
				}
			}
		}

	private suspend fun createProcess(options: CommandExecutionOptions): Process =
		withContext(Dispatchers.IO) {
			Runtime.getRuntime().exec(options.command.toTypedArray())
		}

	private fun CoroutineScope.asyncRead(input: InputStream, consume: suspend (String) -> Unit): Job =
		launch {
			try {
				input.bufferedReader()
					.lineSequence()
					.asFlow()
					.collect { line ->
						consume(line)
					}
			} catch (ex: IOException) {
				logger.warn("Error while reading from process", ex)
				throw ex
			}
		}

	companion object : LogCapability

}

最近我发现在使用该类时, 有时会抛出java.io.IOException: Stream closed异常
异常栈如下:

14:10:38.016 [DefaultDispatcher-worker-117] WARN com.fastonetech.billing.sync.infra.command.CommandExecutorImpl - Error while reading from process
java.io.IOException: Stream closed
	at java.base/java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:168)
	at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:334)
	at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:270)
	at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:313)
	at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:188)
	at java.base/java.io.InputStreamReader.read(InputStreamReader.java:177)
	at java.base/java.io.BufferedReader.fill(BufferedReader.java:162)
	at java.base/java.io.BufferedReader.readLine(BufferedReader.java:329)
	at java.base/java.io.BufferedReader.readLine(BufferedReader.java:396)
	at kotlin.io.LinesSequence$iterator$1.hasNext(ReadWrite.kt:79)
	at kotlinx.coroutines.flow.FlowKt__BuildersKt$asFlow$$inlined$unsafeFlow$5.collect(SafeCollector.common.kt:114)
	at com.fastonetech.billing.sync.infra.command.CommandExecutorImpl$asyncRead$1.invokeSuspend(CommandExecutorImpl.kt:58)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
	at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
	at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)

下面将解决该问题的原因和解决方案.

基于clojure表达式实现更加灵活的数据验证

March 2, 2023 · 13 min read

orange

programmer on jvm platform

数据验证是一个非常常见的需求, 对于java项目来说, 目前jakarta的bean validation已经成为了java中的标准.
其自带了一些常见的数据验证注解, 例如@NotNull, @NotEmpty, @Size等.
这些注解如果遇到复杂的数据验证需求时, 就会显得力不从心. 所以需要一种更加灵活的数据验证方式.
为了满足这种需求, 我们可以通过clojure表达式来实现数据验证.
同时我们需要和现有的bean validation一起使用, 以便于满足现有的业务需求.

当调度服务接口时报错提示cannot execute UPDATE in a read-only transaction问题解决

January 30, 2023 · 4 min read

orange

programmer on jvm platform

前端请求服务相应接口报错, 日志如下

2023-01-30 12:10:14.822  WARN 1 --- [nio-4396-exec-6] o.h.engine.jdbc.spi.SqlExceptionHelper   : SQL Error: 0, SQLState: 25006
2023-01-30 12:10:14.822 ERROR 1 --- [nio-4396-exec-6] o.h.engine.jdbc.spi.SqlExceptionHelper   : ERROR: cannot execute UPDATE in a read-only transaction
	at org.springframework.security.web.authentication.AnonymousAuthen
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:81)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:126)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:115)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:121)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:336)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:81)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:115)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:327)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:684)
	at org.springframework.web.servlet.FrameworkServlet.doPut(FrameworkServlet.java:920)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at jdk.internal.reflect.GeneratedMethodAccessor355.invoke(Unknown Source)
	at com.fastonetech.computecloud.api.regional.software.controller.LaunchableAppController.lastAccessAt(LaunchableAppController.kt:56)
	at com.fastonetech.computecloud.api.regional.software.service.UserSoftwareUsageServiceImpl$$EnhancerBySpringCGLIB$$1caff315.lastAccessAt(<generated>)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:407)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:654)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:711)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:743)
	at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:562)
	at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:281)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$300(JdbcResourceLocalTransactionCoordinatorImpl.java:40)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:183)
	at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:448)
	at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2380)
	at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3212)
	at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:453)
	at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1362)
	at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:99)
	at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:40)
	at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:344)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:475)
	at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:721)
	at org.hibernate.engine.spi.ActionQueue.lambda$executeActions$1(ActionQueue.java:478)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:604)
	at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:201)
	at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3769)
	at org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:3355)
	at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3493)
	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:197)
	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
	at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
	at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:130)
	at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:164)
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:401)
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:481)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:322)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2297)
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2565)
org.postgresql.util.PSQLException: ERROR: cannot execute UPDATE in a read-only transaction

原因

数据库采用主从架构, 由于主库宕机, Load balancer将请求转发到了从库, 从库的事务为read only导致更新失败。

后端相关代码如下

@javax.transaction.Transactional
fun lastAccessAt(id: Long, type: LaunchableAppType): UserSoftwareUsageDTO {
    val userId = authService.currentAsPortal().bind().userId
    val userSoftwareUsage =
        getOrElseCreate(userId, id, type).copy(lastAccessAt = ZonedDateTime.now()).let(repository::save)
    return UserSoftwareUsageDTO(
        userSoftwareUsage.appId,
        userSoftwareUsage.appType,
        userSoftwareUsage.lastAccessAt,
        userSoftwareUsage.collected
    )
}

其采用的javax的@Transactional进行事务控制.
该注解没有提供显式指定数据库事务的读写行为相关属性, 是否是只读或者写入只能由由数据库的默认行为决定.

在PostgreSQL中, 可以通过以下命令来查看默认的读写行为

show default_transaction_read_only;
-- return true if default transaction is read only, false otherwise

解决方案

数据库主机宕机时, 从库升级为主库. 这样可以确保数据库读写不受影响。
java侧需要采用可以显式控制事务是否为readonly的框架, 如spring的@Transactional注解, 该注解提供了readOnly属性, 可以显式控制事务的读写行为。这样可以不隐式依赖数据库的默认行为, 从而在创建事务时提前发现问题。

备注

通过引入防御性复制以避免ConcurrentModificationException

November 24, 2022 · 2 min read

orange

programmer on jvm platform

在consul中修改相关服务的配置时引发ConcurrentModificationException并导致协程任务异常退出.
相关报错如下:

2022-11-24 10:08:27.954  INFO 1 --- [TaskScheduler-1] b.c.PropertySourceBootstrapConfiguration : Located property source: [BootstrapPropertySource {name='bootstrapProperties-config/mgmt-scheduler/'}, BootstrapPropertySource {name='bootstrapProperties-config/application/'}]
2022-11-24 10:08:27.968  INFO 1 --- [TaskScheduler-1] o.s.boot.SpringApplication               : No active profile set, falling back to 1 default profile: "default"
2022-11-24 10:08:27.979  INFO 1 --- [TaskScheduler-1] o.s.boot.SpringApplication               : Started application in 0.244 seconds (JVM running for 7930.38)
Exception in thread "DefaultDispatcher-worker-6" java.util.ConcurrentModificationException
at java.base/java.util.ArrayList$Itr.checkForComodification(ArrayList.java:1013)
at java.base/java.util.ArrayList$Itr.next(ArrayList.java:967)
at com.fastonetech.scheduling.core.ResourceDispatcher$Companion$of$1$1.invokeSuspend(ResourceDispatcher.kt:40)
at com.fastonetech.scheduling.core.ResourceDispatcher$Companion$of$1$1.invoke(ResourceDispatcher.kt)
at com.fastonetech.scheduling.core.ResourceDispatcher$Companion$of$1$1.invoke(ResourceDispatcher.kt)
at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:89)
at kotlinx.coroutines.CoroutineScopeKt.coroutineScope(CoroutineScope.kt:264)
at com.fastonetech.scheduling.core.ResourceDispatcher$Companion$of$1.dispatch(ResourceDispatcher.kt:16)
at com.fastonetech.scheduling.core.ResourceDispatcher$Companion$of$3.schedule(ResourceDispatcher.kt:32)
at com.fastonetech.scheduling.core.ResourceDispatcher$Companion$of$3.schedule(ResourceDispatcher.kt:27)
at com.fastonetech.scheduling.core.ResourceDispatcher$Companion$of$1$1$2$1.invokeSuspend(ResourceDispatcher.kt:19)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.internal.LimitedDispatcher.run(LimitedDispatcher.kt:42)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:95)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:749)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
Suppressed: kotlinx.coroutines.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@19b8b4b6, Dispatchers.IO]

原因

这个问题的原因是被修改的配置映射到了代码中被@ConfigurationProperties注解的类中的一个List类型的属性.
该属性被修改时恰好协程任务正在遍历该属性, 从而导致ConcurrentModificationException异常.

解决方案

每次获取该属性时都进行一次防御性复制, 从而避免ConcurrentModificationException异常

参考资料

在kotlin项目中kotlin-jpa插件导致导致null检查失效

April 7, 2021 · One min read

orange

programmer on jvm platform

kotlin-jpa插件会为data class生成无参构造器，导致非空字段跳过了Null检查

@Entity
class ProjectInfo(
	var name: String,
	var code: String,
	var ownerName: String,
	var applicantName: String,
	var companyCode: String,
	var companyName: String,
	var projectType: ProjectType,
	var submitDate: LocalDateTime = LocalDateTime.now(),
	var planStartDate: LocalDate?,
	var planEndDate: LocalDate?,
	var endDate: String,
	var targetCustomers: Array<String>?,
	var formStatus: ApplicationStatus = ApplicationStatus.DRAFT,
	var projectStatus: ApplicationProjectStatus = ApplicationProjectStatus.DRAFT,
	var comments: String?
) {

	@Id
	@GeneratedValue(strategy = GenerationType.IDENTITY)
	val id: Long = -1
}

原因​

解决方案​

备注​

原因​

解决方案​

参考资料​

原因

解决方案

备注

原因

解决方案

参考资料