History collection too big

[mark at montanasky.net]

Ideally it was to collect enough periodic stats to graph them.  I'll 
admit that every 60 seconds is a bit much.  Do you think increasing it 
to every 5 or 10 minutes would be enough to speed up the API calls?

-Mark

===================================================

Is there a reason you need a 60 second inform interval? I would start by 
backing off your inform interval.

-dan

On Wed, Oct 18, 2017 at 10:03 AM, <mark at montanasky.net> wrote:
We're running version 1.0.? (not sure how to check)

I have parameters.json set to track 6 params:

"InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.1.Stats.EthernetBytesSent" 
: {"track" : true},
"InternetGatewayDevice.WANDevice.1.WANConnectionDevice.1.WANPPPConnection.1.Stats.EthernetBytesReceived" 
: {"track" : true},
"InternetGatewayDevice.WANDevice.1.WANDSLInterfaceConfig.UpstreamNoiseMargin" 
: {"track" : true},
"InternetGatewayDevice.WANDevice.1.WANDSLInterfaceConfig.DownstreamNoiseMargin" 
: {"track" : true},
"InternetGatewayDevice.WANDevice.1.WANDSLInterfaceConfig.UpstreamAttenuation" 
: {"track" : true},
"InternetGatewayDevice.WANDevice.1.WANDSLInterfaceConfig.DownstreamAttenuation" 
: {"track" : true}


We currently have about 1200 devices reporting every 60 seconds.  I have 
a TTL index on the History collection that expires records older than 72 
hours.  Even with this TTL, there are about 8 million records currently 
in the collection, and this makes most API calls to the History 
collection timeout.

Does anybody have any suggestions for keeping the History collection 
from getting so large?  Maybe some way to avoid saving these params on 
EVERY poll?  Maybe just save from a given device once per hour?  Or once 
per half hour?

I'm open to any and all suggestions.

Thanks in advance!