<div dir="ltr">It is reproducible. It happens after a provision script runs and a fault occurs on the device.<div><br></div><div><div><font face="monospace, monospace">$ grep "TypeError" genieacs-cwmp.log -B2</font></div><div><font face="monospace, monospace">2017-05-03T22:47:58.109Z [ERROR] 172.27.240.101 00236a-SR552n-SR552NA084%2D0003269: Channel has faulted; channel="Provisioning" retries=0 faultCode="cwmp.9002" faultMessage="Internal Error"</font></div><div><font face="monospace, monospace">2017-05-03T22:47:58.109Z [ERROR] 172.27.240.101 00236a-SR552n-SR552NA084%2D0003269: Channel has faulted; channel="UpgradeCheck" retries=0 faultCode="cwmp.9002" faultMessage="Internal Error"</font></div><div><font face="monospace, monospace">2017-05-03T22:47:58.285Z [ERROR] Uncaught exception; exceptionName="TypeError" exceptionMessage="Cannot read property 'connection' of undefined" exceptionStack="TypeError: Cannot read property 'connection' of undefined\n at flatten (/usr/lib64/node_modules/genieacs/lib/logger.js:138:63)\n at formatSimple (/usr/lib64/node_modules/genieacs/lib/logger.js:209:3)\n at accessLog (/usr/lib64/node_modules/genieacs/lib/logger.js:289:27)\n at Object.accessInfo (/usr/lib64/node_modules/genieacs/lib/logger.js:296:3)\n at sendAcsRequest (/usr/lib64/node_modules/genieacs/lib/cwmp.js:631:10)\n at /usr/lib64/node_modules/genieacs/lib/cwmp.js:415:16\n at rpcRequest (/usr/lib64/node_modules/genieacs/lib/session.js:988:12)\n at /usr/lib64/node_modules/genieacs/lib/session.js:819:14\n at clear (/usr/lib64/node_modules/genieacs/lib/session.js:1604:12)\n at rpcRequest (/usr/lib64/node_modules/genieacs/lib/session.js:814:12)"</font></div><div><font face="monospace, monospace">--</font></div><div><font face="monospace, monospace">2017-05-03T22:58:55.117Z [ERROR] 172.27.240.101 00236a-SR552n-SR552NA084%2D0003269: Channel has faulted; channel="Provisioning" retries=0 faultCode="cwmp.9002" faultMessage="Internal Error"</font></div><div><font face="monospace, monospace">2017-05-03T22:58:55.117Z [ERROR] 172.27.240.101 00236a-SR552n-SR552NA084%2D0003269: Channel has faulted; channel="UpgradeCheck" retries=0 faultCode="cwmp.9002" faultMessage="Internal Error"</font></div><div><font face="monospace, monospace">2017-05-03T22:58:55.325Z [ERROR] Uncaught exception; exceptionName="TypeError" exceptionMessage="Cannot read property 'connection' of undefined" exceptionStack="TypeError: Cannot read property 'connection' of undefined\n at flatten (/usr/lib64/node_modules/genieacs/lib/logger.js:138:63)\n at formatSimple (/usr/lib64/node_modules/genieacs/lib/logger.js:209:3)\n at accessLog (/usr/lib64/node_modules/genieacs/lib/logger.js:289:27)\n at Object.accessInfo (/usr/lib64/node_modules/genieacs/lib/logger.js:296:3)\n at sendAcsRequest (/usr/lib64/node_modules/genieacs/lib/cwmp.js:631:10)\n at /usr/lib64/node_modules/genieacs/lib/cwmp.js:415:16\n at rpcRequest (/usr/lib64/node_modules/genieacs/lib/session.js:988:12)\n at /usr/lib64/node_modules/genieacs/lib/session.js:819:14\n at clear (/usr/lib64/node_modules/genieacs/lib/session.js:1604:12)\n at rpcRequest (/usr/lib64/node_modules/genieacs/lib/session.js:814:12)"</font></div><div><font face="monospace, monospace">--</font></div><div><font face="monospace, monospace">2017-05-03T23:05:58.956Z [ERROR] 172.27.240.101 00236a-SR552n-SR552NA084%2D0003269: Channel has faulted; channel="Provisioning" retries=0 faultCode="cwmp.9002" faultMessage="Internal Error"</font></div><div><font face="monospace, monospace">2017-05-03T23:05:58.957Z [ERROR] 172.27.240.101 00236a-SR552n-SR552NA084%2D0003269: Channel has faulted; channel="UpgradeCheck" retries=0 faultCode="cwmp.9002" faultMessage="Internal Error"</font></div><div><font face="monospace, monospace">2017-05-03T23:05:59.140Z [ERROR] Uncaught exception; exceptionName="TypeError" exceptionMessage="Cannot read property 'connection' of undefined" exceptionStack="TypeError: Cannot read property 'connection' of undefined\n at flatten (/usr/lib64/node_modules/genieacs/lib/logger.js:138:63)\n at formatSimple (/usr/lib64/node_modules/genieacs/lib/logger.js:209:3)\n at accessLog (/usr/lib64/node_modules/genieacs/lib/logger.js:289:27)\n at Object.accessInfo (/usr/lib64/node_modules/genieacs/lib/logger.js:296:3)\n at sendAcsRequest (/usr/lib64/node_modules/genieacs/lib/cwmp.js:631:10)\n at /usr/lib64/node_modules/genieacs/lib/cwmp.js:415:16\n at rpcRequest (/usr/lib64/node_modules/genieacs/lib/session.js:988:12)\n at /usr/lib64/node_modules/genieacs/lib/session.js:819:14\n at clear (/usr/lib64/node_modules/genieacs/lib/session.js:1604:12)\n at rpcRequest (/usr/lib64/node_modules/genieacs/lib/session.js:814:12)"</font></div><div><font face="monospace, monospace">--</font></div><div><font face="monospace, monospace">2017-05-03T23:09:45.984Z [ERROR] 172.27.240.101 00236a-SR552n-SR552NA084%2D0003269: Channel has faulted; channel="Provisioning" retries=0 faultCode="cwmp.9002" faultMessage="Internal Error"</font></div><div><font face="monospace, monospace">2017-05-03T23:09:45.984Z [ERROR] 172.27.240.101 00236a-SR552n-SR552NA084%2D0003269: Channel has faulted; channel="UpgradeCheck" retries=0 faultCode="cwmp.9002" faultMessage="Internal Error"</font></div><div><font face="monospace, monospace">2017-05-03T23:09:46.172Z [ERROR] Uncaught exception; exceptionName="TypeError" exceptionMessage="Cannot read property 'connection' of undefined" exceptionStack="TypeError: Cannot read property 'connection' of undefined\n at flatten (/usr/lib64/node_modules/genieacs/lib/logger.js:138:63)\n at formatSimple (/usr/lib64/node_modules/genieacs/lib/logger.js:209:3)\n at accessLog (/usr/lib64/node_modules/genieacs/lib/logger.js:289:27)\n at Object.accessInfo (/usr/lib64/node_modules/genieacs/lib/logger.js:296:3)\n at sendAcsRequest (/usr/lib64/node_modules/genieacs/lib/cwmp.js:631:10)\n at /usr/lib64/node_modules/genieacs/lib/cwmp.js:415:16\n at rpcRequest (/usr/lib64/node_modules/genieacs/lib/session.js:988:12)\n at /usr/lib64/node_modules/genieacs/lib/session.js:819:14\n at clear (/usr/lib64/node_modules/genieacs/lib/session.js:1604:12)\n at rpcRequest (/usr/lib64/node_modules/genieacs/lib/session.js:814:12)"</font></div><div><font face="monospace, monospace">--</font></div><div><font face="monospace, monospace">2017-05-04T16:50:33.445Z [ERROR] 172.27.240.101 00236a-SR552n-SR552NA084%2D0003269: Channel has faulted; channel="Provisioning" retries=0 faultCode="cwmp.9002" faultMessage="Internal Error"</font></div><div><font face="monospace, monospace">2017-05-04T16:50:33.445Z [ERROR] 172.27.240.101 00236a-SR552n-SR552NA084%2D0003269: Channel has faulted; channel="UpgradeCheck" retries=0 faultCode="cwmp.9002" faultMessage="Internal Error"</font></div><div><font face="monospace, monospace">2017-05-04T16:50:33.618Z [ERROR] Uncaught exception; exceptionName="TypeError" exceptionMessage="Cannot read property 'connection' of undefined" exceptionStack="TypeError: Cannot read property 'connection' of undefined\n at flatten (/usr/lib64/node_modules/genieacs/lib/logger.js:138:63)\n at formatSimple (/usr/lib64/node_modules/genieacs/lib/logger.js:209:3)\n at accessLog (/usr/lib64/node_modules/genieacs/lib/logger.js:289:27)\n at Object.accessInfo (/usr/lib64/node_modules/genieacs/lib/logger.js:296:3)\n at sendAcsRequest (/usr/lib64/node_modules/genieacs/lib/cwmp.js:631:10)\n at /usr/lib64/node_modules/genieacs/lib/cwmp.js:415:16\n at rpcRequest (/usr/lib64/node_modules/genieacs/lib/session.js:988:12)\n at /usr/lib64/node_modules/genieacs/lib/session.js:819:14\n at clear (/usr/lib64/node_modules/genieacs/lib/session.js:1604:12)\n at rpcRequest (/usr/lib64/node_modules/genieacs/lib/session.js:814:12)"</font></div><div><font face="monospace, monospace">--</font></div><div><font face="monospace, monospace">2017-05-04T17:03:16.880Z [WARN] 172.27.240.20 00236a-96318REF-SR360NA0A4%2D0003196: CPE fault; acsRequestId="1714636915" cpeFaultCode="Server" cpeFaultString="CWMP fault"</font></div><div><font face="monospace, monospace">2017-05-04T17:03:17.113Z [INFO] 172.27.240.20 00236a-96318REF-SR360NA0A4%2D0003196: Script: Firmware_Upgrade - Config files; previous="360_defaults_2017_apr_12.conf" newFile="360_defaults_2017_apr_12.conf"</font></div><div><font face="monospace, monospace">2017-05-04T17:03:17.179Z [ERROR] Uncaught exception; exceptionName="TypeError" exceptionMessage="Cannot read property 'connection' of undefined" exceptionStack="TypeError: Cannot read property 'connection' of undefined\n at /usr/lib64/node_modules/genieacs/lib/cwmp.js:551:60\n at /usr/lib64/node_modules/genieacs/lib/session.js:1980:14\n at /usr/lib64/node_modules/genieacs/lib/db.js:697:12\n at handleCallback (/usr/lib64/node_modules/genieacs/node_modules/mongodb/lib/utils.js:120:56)\n at /usr/lib64/node_modules/genieacs/node_modules/mongodb/lib/collection.js:1062:5\n at /usr/lib64/node_modules/genieacs/node_modules/mongodb-core/lib/connection/pool.js:461:18\n at _combinedTickCallback (internal/process/next_tick.js:73:7)\n at process._tickCallback (internal/process/next_tick.js:104:9)"</font></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, May 5, 2017 at 1:38 PM, Zaid Abdulla <span dir="ltr"><<a href="mailto:zaid@genieacs.com" target="_blank">zaid@genieacs.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On Wed, 2017-05-03 at 15:04 -0800, Dan Morphis wrote:<br>
> Faulty line:<br>
> details.remoteAddress =<br>
> details.sessionContext.<wbr>httpRequest.connection.<wbr>remoteAddress;<br>
><br>
> Uncaught exception; exceptionName="TypeError<br>
> Cannot read property 'connection' of undefined"<br>
> TypeError: Cannot read property 'connection' of undefined<br>
> at flatten<br>
> (/usr/lib64/node_modules/<wbr>genieacs/lib/logger.js:138:63)<br>
> at formatSimple<br>
> (/usr/lib64/node_modules/<wbr>genieacs/lib/logger.js:209:3)<br>
> at accessLog<br>
> (/usr/lib64/node_modules/<wbr>genieacs/lib/logger.js:289:27)<br>
> at Object.accessInfo<br>
> (/usr/lib64/node_modules/<wbr>genieacs/lib/logger.js:296:3)<br>
> at sendAcsRequest<br>
> (/usr/lib64/node_modules/<wbr>genieacs/lib/cwmp.js:631:10)<br>
> at /usr/lib64/node_modules/<wbr>genieacs/lib/cwmp.js:415:16<br>
> at rpcRequest<br>
> (/usr/lib64/node_modules/<wbr>genieacs/lib/session.js:988:<wbr>12)<br>
> at /usr/lib64/node_modules/<wbr>genieacs/lib/session.js:819:14<br>
> at clear<br>
> (/usr/lib64/node_modules/<wbr>genieacs/lib/session.js:1604:<wbr>12)<br>
> at rpcRequest<br>
> (/usr/lib64/node_modules/<wbr>genieacs/lib/session.js:814:<wbr>12)<br>
<br>
</div></div>Couldn't figure out why this might be happening. Any clues? Can you<br>
reproduce it?<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Zaid Abdulla <<a href="mailto:zaid@genieacs.com">zaid@genieacs.com</a>><br>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.genieacs.com">Users@lists.genieacs.com</a><br>
<a href="http://lists.genieacs.com/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.genieacs.com/<wbr>mailman/listinfo/users</a><br>
</font></span></blockquote></div><br></div>