新项目生产测试接口,报错OOM
org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.OutOfMemoryError: Java heap space
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1054)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:897)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.mfw.flight.server.filter.RequestResponseLogFilter.doFilter(RequestResponseLogFilter.java:70)
因为是搜索场景,以为是结果太多导致,
修改结果大小,1000改到200,无效。
修改heap大小,4G改到8G,无效。
决定在请求的时候,实时dump出内存看下。
jmap -dump:live,format=b,file='/tmp/dump.hprof' 1
下载dump文件到本地,安装mat分析工具,打开mat,导入dump文件,查看Leak Suspects
其中a占了2.3G,看着是问题来源,查看stacktrace
定位到buildFullName,查看代码
private String buildFullName(Mdd mdd, Map<Long, Mdd> parentMddMap) {
StringBuilder fullName = new StringBuilder();
fullName.append(mdd.getMddNameChn());
Long parentMddId = mdd.getParentMddId();
int loops = 0;
do {
Mdd currentMdd = parentMddMap.get(parentMddId);
if (Objects.isNull(currentMdd)) {
break;
}
if (currentMdd.getMddTypeCode().reachContinent()) {
break;
}
fullName.append(", ").append(currentMdd.getMddNameChn());
parentMddId = currentMdd.getParentMddId()
} while (Objects.nonNull(parentMddId) && parentMddId != 0L);
while循环!所以可能是这里出现死循环了,加日志,发现存在parent_id=当前ID的情况(脏数据),导致死循环。修复,问题不再复现。
while循环考虑异常情况(不要对数据完全信赖)