RocksDB代码分析——各种option的传递

阅读量: searchstar 2022-03-08 22:22:50
Categories: Tags:
struct Options : public DBOptions, public ColumnFamilyOptions {
struct ImmutableOptions : public ImmutableDBOptions, public ImmutableCFOptions {

Options of column families

Status DB::Open(const Options& options, const std::string& dbname, DB** dbptr) {
  DBOptions db_options(options);
  ColumnFamilyOptions cf_options(options);
  std::vector<ColumnFamilyDescriptor> column_families;
  column_families.push_back(
      ColumnFamilyDescriptor(kDefaultColumnFamilyName, cf_options));
...
    Status s = DB::Open(db_options, dbname, column_families, &handles, dbptr);

即从Options中取出ColumnFamilyOptions并且放到default column family里。

Status DB::Open(const DBOptions& db_options, const std::string& dbname,
                const std::vector<ColumnFamilyDescriptor>& column_families,
                std::vector<ColumnFamilyHandle*>* handles, DB** dbptr) {
  const bool kSeqPerBatch = true;
  const bool kBatchPerTxn = true;
  return DBImpl::Open(db_options, dbname, column_families, handles, dbptr,
                      !kSeqPerBatch, kBatchPerTxn);
}
Status DBImpl::Open(const DBOptions& db_options, const std::string& dbname,
                    const std::vector<ColumnFamilyDescriptor>& column_families,
                    std::vector<ColumnFamilyHandle*>* handles, DB** dbptr,
                    const bool seq_per_batch, const bool batch_per_txn) {
...
      for (auto cf : column_families) {
        ...
            s = impl->CreateColumnFamily(cf.options, cf.name, &handle);
        ...

DBImpl::CreateColumnFamily调用了DBImpl::CreateColumnFamilyImplColumnFamilyOptions是参数。

DBImpl::CreateColumnFamilyImpl执行了versions_->LogAndApply,即调用了VersionSet::LogAndApplyColumnFamilyOptions是参数。

VersionSet::LogAndApply调用了另一个版本的VersionSet::LogAndApplyColumnFamilyOptions是参数。

另一个版本的VersionSet::LogAndApply调用了VersionSet::ProcessManifestWritesColumnFamilyOptions是参数。

VersionSet::ProcessManifestWrites调用了VersionSet::CreateColumnFamilyColumnFamilyOptions是参数。

VersionSet::CreateColumnFamily执行了column_family_set_->CreateColumnFamily,即调用了ColumnFamilySet::CreateColumnFamilyColumnFamilyOptions是参数。然后把返回的ColumnFamilyData返回回去。(这里的version list没搞懂)

ColumnFamilySet::CreateColumnFamily构造了ColumnFamilyData,把db_options_和自己的参数const ColumnFamilyOptions& options传进去。然后把构造好的ColumnFamilyData加入到自己的linked list里,最后返回回去。

ColumnFamilyData的构造函数中,经过一系列操作,把传进来的ColumnFamilyOptionsImmutableDBOptions里的immutable部分放到ImmutableOptions类型的ioptions_里,mutable部分放到MutableCFOptions类型的mutable_cf_options_里。此外,ColumnFamilyData的构造函数中还初始化了compaction_picker_,并将自己的ImmutableOptions类型的ioptions_存一份到LevelCompactionPicker::ioptions_中。

所以这时,Column family的option就保存在VersionSet类型的DBImpl::versions_里的

VersionSet::column_family_set_里的linked list里的

ColumnFamilyData里的

ImmutableOptions类型的ioptions_MutableCFOptions类型的mutable_cf_options_里。

Options of DB

Status DB::Open(const Options& options, const std::string& dbname, DB** dbptr) {
  DBOptions db_options(options);
...
  Status s = DB::Open(db_options, dbname, column_families, &handles, dbptr);
Status DB::Open(const DBOptions& db_options, const std::string& dbname,
                const std::vector<ColumnFamilyDescriptor>& column_families,
                std::vector<ColumnFamilyHandle*>* handles, DB** dbptr) {
  const bool kSeqPerBatch = true;
  const bool kBatchPerTxn = true;
  return DBImpl::Open(db_options, dbname, column_families, handles, dbptr,
                      !kSeqPerBatch, kBatchPerTxn);
}
  DBImpl* impl = new DBImpl(db_options, dbname, seq_per_batch, batch_per_txn);

DBImpl的构造函数中,把DBOptions拆成了mutable_db_options_immutable_db_options_。此外,还初始化了versions_,把ImmutableDBOptions存入了DBImpl::versions_.column_family_set_.db_options_,并且把column_family_memtables_.column_family_set_指针设置为versions_.column_family_set_

写入路径上的option传递

DBImpl::WriteImpl中,通过ColumnFamilyMemTablesImpl类型的column_family_memtables能访问到ImmutableDBOptions和所有ColumnFamilyOptionscolumn_family_memtables作为参数传给WriteBatchInternal::InsertIntomemtables形参,类型转换为ColumnFamilyMemTables

WriteBatchInternal::InsertInto中,把memtables保存到inserter.cf_mems_,此时ColumnFamilyMemTablesImplinserter.cf_mems_inserter传给了WriteBatch::Iteratehandler形参。

WriteBatch::Iterate中,ColumnFamilyMemTablesImplhandler.cf_mems_。调用WriteBatchInternal::Iteratehandler作为参数,名字不变。

WriteBatchInternal::Iterate中,ColumnFamilyMemTablesImplhandler.cf_mems_。执行handler->PutCF,即调用MemTableInserter::PutCF

MemTableInserter::PutCF中,ColumnFamilyMemTablesImplcf_mems_。调用MemTableInserter::PutCFImpl

MemTableInserter::PutCFImpl中,ColumnFamilyMemTablesImplcf_mems_。先调用MemTableInserter::SeekToColumnFamily,在里面执行cf_mems_->Seek(column_family_id),即调用ColumnFamilyMemTablesImpl::Seek,在里面设置current_

MemTableInserter::CheckMemtableFull中,执行cfd = cf_mems_->current(),取出当前的column family的ColumnFamilyData,然后可能执行flush_scheduler_->ScheduleWork(cfd)。注意,通过ColumnFamilyData可以访问到ImmutableOptionsMutableCFOptions

然后根据 RocksDB代码分析——写入流程,一波操作之后ColumnFamilyData作为flush任务的载体被放到了FlushRequest里,进而被放进了DBImpl::flush_queue_里。

然后根据 RocksDB代码分析——Flush流程DBImpl::BackgroundFlush调用DBImpl::PopFirstFromFlushQueueDBImpl::flush_queue_中取出FlushRequest flush_req,从中再取出ColumnFamilyData。一波操作之后ColumnFamilyData可能作为compaction任务的载体被放进DBImpl::compaction_queue_

然后根据 RocksDB代码分析——Compaction流程DBImpl::BackgroundCompaction可能调用DBImpl::PickCompactionFromQueuecompaction_queue_里取出一个ColumnFamilyData *cfd,然后调用cfd->PickCompaction,得到Compaction *

ColumnFamilyData::PickCompaction执行了compaction_picker_->PickCompaction,即调用了LevelCompactionPicker::PickCompaction

LevelCompactionPicker::PickCompaction中构造了LevelCompactionBuilder builder,将ImmutableOptions类型的LevelCompactionPicker::ioptions_存入LevelCompactionBuilder::ioptions_

由上文可知,LevelCompactionPicker::ioptions_是在构造ColumnFamilyData时初始化的,其ioptions_来自ColumnFamilyData::ioptions_