History collection too big

mark at montanasky.net mark at montanasky.net
Wed Oct 18 14:03:58 EDT 2017


We're running version 1.0.? (not sure how to check)

I have parameters.json set to track 6 params:

"InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.1.Stats.EthernetBytesSent" 
: {"track" : true},
"InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.1.Stats.EthernetBytesReceived" 
: {"track" : true},
"InternetGatewayDevice.WANDevice.1.WANDSLInterfaceConfig.UpstreamNoiseMargin" 
: {"track" : true},
"InternetGatewayDevice.WANDevice.1.WANDSLInterfaceConfig.DownstreamNoiseMargin" 
: {"track" : true},
"InternetGatewayDevice.WANDevice.1.WANDSLInterfaceConfig.UpstreamAttenuation" 
: {"track" : true},
"InternetGatewayDevice.WANDevice.1.WANDSLInterfaceConfig.DownstreamAttenuation" 
: {"track" : true}


We currently have about 1200 devices reporting every 60 seconds.  I have 
a TTL index on the History collection that expires records older than 72 
hours.  Even with this TTL, there are about 8 million records currently 
in the collection, and this makes most API calls to the History 
collection timeout.

Does anybody have any suggestions for keeping the History collection 
from getting so large?  Maybe some way to avoid saving these params on 
EVERY poll?  Maybe just save from a given device once per hour?  Or once 
per half hour?

I'm open to any and all suggestions.

Thanks in advance!


More information about the Users mailing list