Trace

新项目生产测试接口,报错OOM

org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.OutOfMemoryError: Java heap space
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1054)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:942)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1005)
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:897)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:882)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at com.mfw.flight.server.filter.RequestResponseLogFilter.doFilter(RequestResponseLogFilter.java:70)

因为是搜索场景,以为是结果太多导致,

修改结果大小,1000改到200,无效。

修改heap大小,4G改到8G,无效。

决定在请求的时候,实时dump出内存看下。

jmap -dump:live,format=b,file='/tmp/dump.hprof' 1

下载dump文件到本地,安装mat分析工具,打开mat,导入dump文件,查看Leak Suspects

Untitled

其中a占了2.3G,看着是问题来源,查看stacktrace

Untitled

定位到buildFullName,查看代码

private String buildFullName(Mdd mdd, Map<Long, Mdd> parentMddMap) {
        StringBuilder fullName = new StringBuilder();
        fullName.append(mdd.getMddNameChn());

        Long parentMddId = mdd.getParentMddId();
        int loops = 0;
        do {
            Mdd currentMdd = parentMddMap.get(parentMddId);
            if (Objects.isNull(currentMdd)) {
                break;
            }
            if (currentMdd.getMddTypeCode().reachContinent()) {
                break;
            }

            fullName.append(", ").append(currentMdd.getMddNameChn());

            parentMddId = currentMdd.getParentMddId()

        } while (Objects.nonNull(parentMddId) && parentMddId != 0L);

while循环!所以可能是这里出现死循环了,加日志,发现存在parent_id=当前ID的情况(脏数据),导致死循环。修复,问题不再复现。

反思

while循环考虑异常情况(不要对数据完全信赖)