ynkmr/Sourcetrail
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

logic: Improvements in indexing to avoid OOM (issue #471)

Browse Source 
* indexer processes wait if already too many storages queued
* storage fetcher waits if already too many storages queued
* increased size estimation for shared memory of indexer commands
* Added MessageFilterNewErrors to combine messages for new errors

bug id = 471
This commit is contained in:
Eberhard Graether
2017-10-12 17:15:46 +02:00
parent 1e38ce53c4
commit af626143a5
10 changed files with 74 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/lib/data/indexer/interprocess/InterprocessIndexer.cpp
+14
View File
@@ -33,6 +33,20 @@ void InterprocessIndexer::work()
LOG_INFO_STREAM(<< m_processId << " fetched indexer command for \"" << indexerCommand->getSourceFilePath().str() << "\"");
LOG_INFO_STREAM(<< m_processId << " indexer commands left: " << (m_interprocessIndexerCommandManager.indexerCommandCount() + 1));
while (true)
{
size_t storageCount = m_interprocessIntermediateStorageManager.getIntermediateStorageCount();
if (storageCount < 10)
{
break;
}
LOG_INFO_STREAM(<< m_processId << " waits, too many intermediate storages: " << storageCount);
const int SLEEP_TIME_MS = 200;
std::this_thread::sleep_for(std::chrono::milliseconds(SLEEP_TIME_MS));
}
LOG_INFO_STREAM(<< m_processId << " updating indexer status with currently indexed filepath");
m_interprocessIndexingStatusManager.startIndexingSourceFile(indexerCommand->getSourceFilePath());