概述
在Android 11中，引入了IORap ，这是一个新功能，可大大缩短应用程序的启动时间。
IORap 通过预测将需要哪些I / O并提前进行来减少应用程序启动时间。许多应用程序在启动时需要访问I/O.
很多时间会因为阻塞I / O而导致应用程序启动慢。预取数据之后，应用程序几乎可以从pagecache 中立即访问该数据，
从而大大减少了应用程序启动延迟。
框架
代码路径
frameworks/base/startop/iorap
一个java层服务，跟iorapd进行通信；主要负责应用状态的监听工作，状态发生变化或者job 触发时，通知iorapd 进程进行核心处理。
监听包括：ODEX 发生更新，activity launch 进度发生变化，job service 用以触发iorapd compile tracing files。这里重点说一下job service，
android 中job service 为后台普通service，为了完成一些优先级不高的任务，而这里的compile task也是使用job service，而且是在charging
状态下进行，且24小时内只做一次任务。
system/iorap/
//iorapd.rc用于启动/system/bin/iorapd进程//system/iorap/src/perfetto主要是收集perfetto trace，生成perfetto_trace.pb //system/iorap/src/compiler主要是生成预取列表。将perfetto_trace.pb转换成comlplied_trace.pb//system/iorap/src/db使用sqlite3实现的一个数据库, 数据保存在/data/misc/iorapd//system/iorap/src/inode2filename将inode转为真正filename的功能//system/iorap/src/prefetcher主要是实现预读取功能//system/iorap/src/manageriorapd主要实现逻辑，事件驱动的一个模型，跟java层服务进行互动，并且驱动各个模块进行工作。 |
工作流程
1、收集perfetto trace。当应用第一次启动时，通过perfetto 产生trace。该trace记录了内核pagecache页面的删除/添加(来自ftrace )。
在应用程序的前几次冷运行中，将进行perfetto跟踪以获取pagecache丢失事件。trace产生后会通知perfetto 的producer 进行记录，
并保存在perfetto_trace.pb 文件中。研究表明，启动时进行perfetto跟踪的开销可以忽略不计。
2、生成预取列表。基于上面的 perfetto trace，IORap 会在设备空闲时， IORap 会根据 perfetto trace 分析 mm_pagemap 事件，并将结果
(inode、偏移量、长度) 转换为 (名称、偏移量、长度)，然后将数据存储在预取列表中，预取列表是一个 protobuf 文件。
3、预读取。生成预取列表之后，在新的一次app 冷启动时，app launch的作为触发条件，会通知iorapd 进行预取工作，将之前解析好的
protobuf 传入prefetcherd 进程，并通过系统调用（例如posix_fadvise）进行内核端的预取工作
4、更新预取列表。预取列表不会永久存在，当APP更新后，APP的数据会和之前数据有差异，这会使预取列表过时，过时的预取列表将被删除，
并且开始新一轮的 perfetto trace。
工作流程详解
服务启动
IorapForwardingService服务启动
frameworks/base/startop/iorap/src/com/google/android/startop/iorap/IorapForwardingService.java
IorapForwardingService是由system server中启动的，主要是做了三件事
- 注册AppLaunchObserver
- 注册DexOptPackagesUpdated
- 启动IorapdJobService
服务启动使能
iorap_perfetto_enable和iorap_readahead_enable需要设置为true
protected boolean isIorapEnabled() { // These two mendel flags should match those in iorapd native process // system/iorapd/src/common/property.h boolean isTracingEnabled = getMendelFlag("iorap_perfetto_enable", "iorapd.perfetto.enable", false); boolean isReadAheadEnabled = getMendelFlag("iorap_readahead_enable", "iorapd.readahead.enable", false); // Same as the property in iorapd.rc -- disabling this will mean the 'iorapd' binder process // never comes up, so all binder connections will fail indefinitely. return IS_ENABLED && (isTracingEnabled || isReadAheadEnabled); } |
onStart
配置相关信息和连接底层的iorap服务
@Overridepublic void onStart() { ... retryConnectToRemoteAndConfigure(/*attempts*/0);} |
retryConnectToRemoteAndConfigure-->connectToRemoteAndConfigure→connectToRemoteAndConfigureLocked
- 连接底层iorap服务
- 注册监听AppLaunchObserver、DexOptPackagesUpdated。后面详细介绍这两个
private boolean connectToRemoteAndConfigureLocked() { ... // Connect to the native binder service. mIorapRemote = provideIorapRemote(); ... invokeRemote(mIorapRemote, (IIorap remote) -> remote.setTaskListener(new RemoteTaskListener()) ); registerInProcessListenersLocked(); ... } |
private void registerInProcessListenersLocked() { ... ActivityMetricsLaunchObserverRegistry launchObserverRegistry = provideLaunchObserverRegistry(); launchObserverRegistry.registerLaunchObserver(mAppLaunchObserver); launchObserverRegistry.registerLaunchObserver(mEventSequenceValidator); BackgroundDexOptService.addPackagesUpdatedListener(mDexOptPackagesUpdated); ... } |
onBootPhase
主要启动IorapdJobService
@Overridepublic void onBootPhase(int phase) { ... mJobService = new IorapdJobService(getContext()); ...} |
启动iorapd的jobserver服务，并且设置了启动的条件。原生代码中是设置在charging状态下进行，且24小时内只做一次任务。
public IorapdJobService(Context context) { ... IORAPD_COMPONENT_NAME = new ComponentName(context, IorapdJobServiceProxy.class); JobInfo.Builder builder = new JobInfo.Builder(JOB_ID_IORAPD, IORAPD_COMPONENT_NAME); builder.setPeriodic(JOB_INTERVAL_MS); builder.setPrefetch(true); builder.setRequiresCharging(true); builder.setRequiresDeviceIdle(true); builder.setRequiresStorageNotLow(true); IORAPD_JOB_INFO = builder.build(); JobScheduler js = context.getSystemService(JobScheduler.class); js.schedule(IORAPD_JOB_INFO); ...} |
执行jobserver任务：onStartJob→runIdleCompilationAsync
调用iorapd中的onJobScheduledEvent 进complize apk，把perfetto_trace.pb 编译生成complized_trace.pb列表
iorapd服务启动
system/iorap/iorapd.rc
# Start iorapd when either prefetching or tracing is enabled.on property:persist.device_config.runtime_native_boot.iorap_perfetto_enable=true && property:ro.iorapd.enable=true start iorapd on property:persist.device_config.runtime_native_boot.iorap_readahead_enable=true && property:ro.iorapd.enable=true start iorapd |
需要设置ro.iorapd.enable、persist.device_config.runtime_native_boot.iorap_perfetto_enable或者
persist.device_config.runtime_native_boot.iorap_readahead_enable为true。
收集Trace和预读取
frameworks/base/startop/iorap/src/com/google/android/startop/iorap/IorapForwardingService.java
上面服务启动中，已经介绍，服务启动时注册AppLaunchObserver，监听activity的启动。AppLaunchObserver定义了下面接口，
主要是调用了iorapd服务中对应接口实现功能
onIntentStarted--->iorapd::onAppLaunchEvent(AppLaunchEvent.IntentStarted)onIntentFailed--->iorapd::onAppLaunchEvent(AppLaunchEvent.IntentFailed)onActivityLaunched--->iorapd::onAppLaunchEvent(AppLaunchEvent.ActivityLaunched)onActivityLaunchCancelled--->iorapd::onAppLaunchEvent(AppLaunchEvent.ActivityLaunchCancelled)onActivityLaunchFinished--->iorapd::onAppLaunchEvent(AppLaunchEvent.ActivityLaunchFinished) |
system/iorap/src/manager/event_manager.cc
OnAppLaunchEventbool EventManager::OnAppLaunchEvent(RequestId request_id, const AppLaunchEvent& event) { return impl_->OnAppLaunchEvent(request_id, event);} |
bool OnAppLaunchEvent(RequestId request_id, const AppLaunchEvent& event) { ... AppLaunchEvent overwrite_event{}; AppLaunchEventDefender::Result result = app_launch_event_defender_.OnAppLaunchEvent(request_id, event, /*out*/&overwrite_event); ...} |
struct AppLaunchEventDefender { ... Result OnAppLaunchEvent(binder::RequestId request_id, const binder::AppLaunchEvent& event, binder::AppLaunchEvent* overwrite) { ... switch (last_event_type_) { case Type::kUninitialized: case Type::kIntentFailed: case Type::kActivityLaunchCancelled: case Type::kReportFullyDrawn: { ... } case Type::kIntentStarted: { ... } case Type::kActivityLaunched: { ... } case Type::kActivityLaunchFinished: { ... } } }}; |
具体执行的任务，在OnNewEvent中执行
void OnNewEvent(const AppLaunchEvent& event) { ... switch (event.type) { case Type::kIntentStarted: ... case Type::kIntentFailed: ... case Type::kActivityLaunched: { ... AppComponentName component_name = AppComponentName::FromString(title); component_name = component_name.Canonicalize(); component_name_ = component_name; ... AppLaunchEvent::Temperature temperature = event.temperature; temperature_ = temperature; //判断是否是冷启动，冷启动才需要进行预读取 if (temperature != AppLaunchEvent::Temperature::kCold) { LOG(INFO) << "AppLaunchEventState#OnNewEvent don't trace due to non-cold temperature"; } else if (!IsTracing() && !IsReadAhead()) { // and the temperature is Cold. //StartReadAhead进行预读取 if (allowed_readahead_ && !IsReadAhead()) { StartReadAhead(sequence_id_, component_name); } //进行perfetto trace if (allowed_tracing_ && !IsTracing() && !IsReadAhead()) { rx_lifetime_ = StartTracing(std::move(component_name)); } } else { ... } break; } case Type::kActivityLaunchFinished: ... FinishReadAhead(); break; case Type::kActivityLaunchCancelled: ... AbortTrace(); AbortReadAhead(); break; case Type::kReportFullyDrawn: ... default: }} |
收集Trace
StartTracing
- 调用perfetto/rx_producer.cc 中的CreateTraceStream 进行trace
- 创建保存trace文件的目录，目录在/data/misc/iorapd/，比如/data/misc/iorapd/com.ctg.itrdc.file/1000002/dev.dworks.apps.anexplorer.DocumentsActivity/raw_traces
- 把读取到的trace写入到perfetto_trace.pb文件中
std::optional<rxcpp::composite_subscription> StartTracing( AppComponentName component_name) { ... auto /*observable<PerfettoTraceProto>*/ trace_proto_stream = perfetto_factory_->CreateTraceStream(perfetto_commands); db::PerfettoTraceFileModel file_model = db::PerfettoTraceFileModel::CalculateNewestFilePath(versioned_component_name); std::string file_path = file_model.FilePath(); ScopedFormatTrace atrace_write_to_file(ATRACE_TAG_ACTIVITY_MANAGER, "Perfetto Write Trace To File %s", file_path.c_str()); //创建文件夹 if (!file_model.MkdirWithParents()) { LOG(ERROR) << "Cannot save TraceBuffer; failed to mkdirs " << file_path; return; } //把trace文件写入到文件中 if (!trace_proto.WriteFullyToFile(file_path)) { LOG(ERROR) << "Failed to save TraceBuffer to " << file_path; } else { LOG(INFO) << "Perfetto TraceBuffer saved to file: " << file_path; ScopedFormatTrace atrace_update_raw_traces_table( ATRACE_TAG_ACTIVITY_MANAGER, "update raw_traces table history_id = %d", history_id); db::DbHandle db{db::SchemaModel::GetSingleton()}; std::optional<db::RawTraceModel> raw_trace = db::RawTraceModel::Insert(db, history_id, file_path); } }, is_tracing_ = true; return lifetime; }/ |
system/iorap/src/perfetto/rx_producer.cc
CreateTraceStream→CreatePerfettoStream
主要调用perfetto/perfetto_consumer.cc中的StartTracing、ReadTrace等接口
static auto/*observable<PerfettoTraceProto>*/ CreatePerfettoStream(rxcpp::observable<PerfettoStreamCommand> input, std::shared_ptr<PerfettoConsumer> perfetto_consumer, const ::perfetto::protos::TraceConfig& trace_config) { // XX: should I also take a scheduler for input here??? auto /*observable<PerfettoStateChange>*/ perfetto_states = CreatePerfettoStateStream(trace_config, perfetto_consumer); ... return input .flat_map( [](std::tuple<PerfettoStreamCommand, PerfettoStateChange> p) { auto& [command, state_change] = p; LOG(INFO) << "CreatePerfettoStream#combine(" << command << "," << state_change << ")"; if (command == PerfettoStreamCommand::kShutdown) { state_change.GetConsumer()->Destroy(state_change.GetHandle()); } else if (command == PerfettoStreamCommand::kStartTracing && state_change.state == State::kConfigured) { state_change.GetConsumer()->StartTracing(state_change.GetHandle()); } else if (command == PerfettoStreamCommand::kStopTracing && state_change.state == State::kTraceEnded) { ::perfetto::consumer::TraceBuffer trace_buffer = state_change.GetConsumer()->ReadTrace(state_change.GetHandle()); PerfettoTraceProto wire_proto{trace_buffer.begin, trace_buffer.size}; return rxcpp::observable<>::just(std::move(wire_proto)).as_dynamic(); } return rxcpp::observable<>::empty<PerfettoTraceProto>().as_dynamic(); } );} |
预读取
StartReadAhead
- 获取compile生成的预取列表。是通过包名和版本号读取，路径比如：/data/misc/iorapd/com.ctg.itrdc.file/1000002/dev.dworks.apps.anexplorer.DocumentsActivity/compiled_traces/compiled_trace.pb
- 执行预读取。调用/prefetcher/read_ahead.cc中 read_ahead_→BeginTask进行预读取
void StartReadAhead(size_t id, const AppComponentName& component_name) { ... std::optional<std::string> file_path = GetCompiledTrace(component_name); if (!file_path) { LOG(INFO) << "Cannot find a compiled trace."; return; } prefetcher::TaskId task{id, *file_path}; read_ahead_->BeginTask(task); // TODO: non-void return signature? read_ahead_task_ = std::move(task); } |
生成预读取列表
Job server的启动在服务启动一节，已经介绍，最终调用到manager/event_manager.cc中的OnJobScheduledEvent
bool OnJobScheduledEvent(RequestId request_id, const JobScheduledEvent& event) { ... job_scheduled_event_subject_.OnNext(std::move(request_id), event); return true; // No errors. } |
Job初始化时，初始化函数InitializeRxGraphForJobScheduledEvents，设置了job处理的任务，最终调用StartMaintenance→CompileSingleAppOnDevice
system/iorap/src/maintenance/controller.cc
bool CompileSingleAppOnDevice(const db::DbHandle& db, const ControllerParameters& params, const std::string& package_name) { std::vector<db::PackageModel> packages = db::PackageModel::SelectByName(db, package_name.c_str()); bool ret = true; for (db::PackageModel package : packages) { if (!CompilePackage(db, package.name, package.version, params)) { ret = false; } } return ret;} |
CompilePackage通过app包名和版本号，查找包db所在路径。然后调用CompileActivity compile 所有activity
// Compiled the perfetto traces for activities in an package.bool CompilePackage(const db::DbHandle& db, const std::string& package_name, int version, const ControllerParameters& params) { ... std::optional<db::PackageModel> package = db::PackageModel::SelectByNameAndVersion(db, package_name.c_str(), version); ... std::vector<db::ActivityModel> activities = db::ActivityModel::SelectByPackageId(db, package->id); bool ret = true; for (db::ActivityModel activity : activities) { if (!CompileActivity(db, package->id, package->name, activity.name, version, params)) { ret = false; } } return ret;} |
- 如果complized_trace.pb文件存在,则直接返回。complized_trace.pb路径在/data/misc/iorapd/com.ctg.itrdc.file/1000002/dev.dworks.apps.anexplorer.DocumentsActivity/compiled_traces/compiled_trace.pb
- 如果complized_trace.pb文件不存在，则通过StartViaFork 启动iorap.cmd.compiler进程。
// Compiled the perfetto traces for an activity.bool CompileActivity { db::CompiledTraceFileModel output_file = CalculateNewestFilePath(package_name, activity_name, version); std::string file_path = output_file.FilePath(); //complized_trace.pb文件如果存在,则直接返回 if (!params.recompile) { if (std::filesystem::exists(file_path)) { ... db::VersionedComponentName vcn{package_name, activity_name, version}; std::optional<db::PrefetchFileModel> prefetch_file = db::PrefetchFileModel::SelectByVersionedComponentName(db, vcn); if (prefetch_file) { return true; } else { ... } } } std::optional<db::ActivityModel> activity = db::ActivityModel::SelectByNameAndPackageId(db, activity_name.c_str(), package_id); ... int activity_id = activity->id; std::vector<db::AppLaunchHistoryModel> histories = db::AppLaunchHistoryModel::SelectActivityHistoryForCompile(db, activity_id); { std::vector<compiler::CompilationInput> perfetto_traces = GetPerfettoTraceInfo(db, histories); ... ScopedFormatTrace atrace_compile_fork(ATRACE_TAG_PACKAGE_MANAGER, "Fork+exec iorap.cmd.compiler", activity_name.c_str()); if (!StartViaFork(compiler_params)) { ... } } ...} |
system/iorap/src/compiler
iorap.cmd.compiler进程的main函数，主要是调用PerformCompilation函数
- ApplyBlacklistToPageCacheEvents中调用iorap.inode2filename中的FindFilenamesFromInodes通过inode 查找文件名。
- serialize::ProtobufIO::WriteFully把 (inode、偏移量、长度) 转换为 (名称、偏移量、长度)，然后将数据存储在预取列表 protobuf 文件中。
bool PerformCompilation(std::vector<CompilationInput> perfetto_traces, std::string output_file_name, bool output_proto, std::optional<std::string> blacklist_filter, inode2filename::InodeResolverDependencies dependencies) { auto trace_protos = ReadPerfettoTraceProtos(std::move(perfetto_traces)); auto resolved_events = ResolvePageCacheEntriesFromProtos(std::move(trace_protos), std::move(dependencies)); auto filtered_events = ApplyBlacklistToPageCacheEvents(std::move(resolved_events), blacklist_filter); auto compiled_events = CompilePageCacheEvents(std::move(filtered_events)); ... if (output_proto) { LOG(INFO) << "compiler: WriteFully to begin into " << output_file_name; ::google::protobuf::MessageLite& message = *trace_file_proto.get(); if (auto res = serialize::ProtobufIO::WriteFully(message, output_file_name); !res) { errno = res.error(); PLOG(ERROR) << "compiler: Failed to write protobuf to file: " << output_file_name; return false; } else { LOG(INFO) << "compiler: Wrote protobuf " << output_file_name; } } |
更新预取列表
startop/iorap/src/com/google/android/startop/iorap/IorapForwardingService.java
服务启动时注册了DexOptPackagesUpdated，DexOptPackagesUpdated会监听应用包的更新
private class DexOptPackagesUpdated implements BackgroundDexOptService.PackagesUpdatedListener { @Override public void onPackagesUpdated(ArraySet<String> updatedPackages) { String[] updated = updatedPackages.toArray(new String[0]); for (String packageName : updated) { Log.d(TAG, "onPackagesUpdated: " + packageName); invokeRemote(mIorapRemote, (IIorap remote) -> remote.onDexOptEvent(RequestId.nextValueForSequence(), DexOptEvent.createPackageUpdate(packageName)) ); } } } |
最终调用到event_manager.ccp中的OnDexOptEvent
bool OnDexOptEvent(RequestId request_id, const DexOptEvent& event) { ... return PurgePackage(event.package_name); } |
PurgePackage调用db中的CleanUpFilesForPackage清除掉过时的db数据
bool PurgePackage(const std::string& package_name) { db::DbHandle db{db::SchemaModel::GetSingleton()}; db::CleanUpFilesForPackage(db, package_name); LOG(INFO) << "PurgePackage: " << package_name; return true; } |
iorap服务监控包更新
system/iorap/src/binder/package_change_observer.cc
android::binder::Status PackageChangeObserver::onPackageChanged( const android::content::pm::PackageChangeEvent& event) { LOG(INFO) << "Received PackageChangeObserver::onPackageChanged"; if (event_manager_->OnPackageChanged(event)) { return android::binder::Status::ok(); } else { return android::binder::Status::fromStatusT(android::BAD_VALUE); }} bool OnPackageChanged(const android::content::pm::PackageChangeEvent& event) { LOG(INFO) << "Received " << event; if (event.isDeleted) { return true; } // Update the version map. if (version_map_->Update(event.packageName, event.version)) { return true; } db::DbHandle db{db::SchemaModel::GetSingleton()}; db::CleanUpFilesForPackage(db, event.packageName, event.version); return true; } |
