<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://lrita.github.io/feed.xml" rel="self" type="application/atom+xml" /><link href="https://lrita.github.io/" rel="alternate" type="text/html" /><updated>2025-08-23T19:34:10+08:00</updated><id>https://lrita.github.io/feed.xml</id><title type="html">源代码</title><subtitle>lrita&apos;s blog</subtitle><author><name>Neal Hu</name></author><entry><title type="html">在宕机后修复丢失的Chrome中打开的页面</title><link href="https://lrita.github.io/2024/04/24/chrome-recovery-tab-after-crash/" rel="alternate" type="text/html" title="在宕机后修复丢失的Chrome中打开的页面" /><published>2024-04-24T00:00:00+08:00</published><updated>2024-04-24T00:00:00+08:00</updated><id>https://lrita.github.io/2024/04/24/chrome-recovery-tab-after-crash</id><content type="html" xml:base="https://lrita.github.io/2024/04/24/chrome-recovery-tab-after-crash/"><![CDATA[<p>当机器宕机以后，重新打开Chrome浏览器时，会提示重新打开<code class="language-plaintext highlighter-rouge">最近关闭的标签页</code>，从而恢复之前的记录，或者可以通过按快捷键<code class="language-plaintext highlighter-rouge">Cmd + Shift + T (Mac)</code>/<code class="language-plaintext highlighter-rouge">Control + Shift + T (Windows)</code>来进行。</p>

<p>但是有的时候宕机之后，Chrome内部损坏，可能会丢失<code class="language-plaintext highlighter-rouge">最近关闭的标签页</code>这个数据，从而造成无法使用Chrome前台功能正常恢复。</p>

<p>有时打开的标签页可能已经数日，已经无法通过浏览器历史召回，此时就可以通过以下方法进行找回：</p>

<h3 id="首先找到会话数据">首先找到会话数据</h3>

<p>参考这个<a href="https://source.chromium.org/chromium/chromium/src/+/e27098ba216a56402f42f98c1689c6bfe139a4c0">Chrome代码提交</a>透露出的信息，在Chrome每次创建一个会话时，创建一个记录当前会话页面的<code class="language-plaintext highlighter-rouge">Session_${timestamp}</code>文件，当有<code class="language-plaintext highlighter-rouge">最近关闭的标签页</code>数据时，会创建一个<code class="language-plaintext highlighter-rouge">Tabs_${timestamp}</code>文件。这些文件被存储在：</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># MacOS在</span>
~/Library/Application<span class="se">\ </span>Support/Google/Chrome/Default/Sessions
<span class="c"># 当Chrome升级时，会备份在这，当需要恢复早先的记录可以考虑使用</span>
~/Library/Application<span class="se">\ </span>Support/Google/Chrome/Snapshots/123.0.6312.124/Default/Sessions

<span class="c"># 注意：以下2个未验证，参考 https://chromium.googlesource.com/chromium/src/+/HEAD/docs/user_data_dir.md#user-cache-directory</span>
<span class="c"># Windows在</span>
C:<span class="se">\U</span>sers[USERNAME]<span class="se">\A</span>ppData<span class="se">\L</span>ocal<span class="se">\G</span>oogle<span class="se">\C</span>hrome<span class="se">\U</span>ser Data<span class="se">\D</span>efault<span class="se">\S</span>essions
<span class="c"># Linux在</span>
/home/<span class="nv">$USER</span>/.config/google-chrome/Default/Sessions
</code></pre></div></div>

<h3 id="解析出tab对应的url">解析出Tab对应的URL</h3>

<p>当我们找到这些文件时，我们直接打开阅读，会发现是乱码(二进制文件)，这是Chrome的<code class="language-plaintext highlighter-rouge">SNSS</code>文件存储格式，需要对应的工具进行解析，经过一番查找，大部分工具都是过时，不能支持最新版的Chrome，只有<a href="https://github.com/cclgroupltd/ccl_chrome_indexeddb">ccl_chrome_indexeddb</a>这个工具还在迭代中，可以支持。</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># 下载工具：</span>
git clone https://github.com/cclgroupltd/ccl_chrome_indexeddb.git
<span class="nb">cd </span>ccl_chrome_indexeddb
<span class="c"># 提取URL到1.txt</span>
python3 ./ccl_chromium_snss2.py ~/Library/Application<span class="se">\ </span>Support/Google/Chrome/Default/Sessions/Tabs_13355542992273182 | rg <span class="nt">-o</span> <span class="nt">--pcre2</span> <span class="s2">"(?&lt;= url=')https?:</span><span class="se">\/\/</span><span class="s2">(www</span><span class="se">\.</span><span class="s2">)?[-a-zA-Z0-9@:%._</span><span class="se">\+</span><span class="s2">~#=]{1,256}</span><span class="se">\.</span><span class="s2">[a-zA-Z0-9()]{1,6}</span><span class="se">\b</span><span class="s2">([-a-zA-Z0-9()@:%_</span><span class="se">\+</span><span class="s2">.~#?&amp;//=]*)(?=')"</span>  <span class="o">&gt;</span> 1.txt
<span class="c"># 查看结果</span>
<span class="nb">cat </span>1.txt
</code></pre></div></div>

<p>然后我们就可以从提取到的标签页对应的链接来恢复页面了。</p>]]></content><author><name>Neal Hu</name></author><category term="tool" /><summary type="html"><![CDATA[tool chrome recovery miss tab]]></summary></entry><entry><title type="html">在 gitlab 中配置A项目触发B项目的 pipeline</title><link href="https://lrita.github.io/2024/03/20/gitlab-trigger-other-repo-pipeline/" rel="alternate" type="text/html" title="在 gitlab 中配置A项目触发B项目的 pipeline" /><published>2024-03-20T00:00:00+08:00</published><updated>2024-03-20T00:00:00+08:00</updated><id>https://lrita.github.io/2024/03/20/gitlab-trigger-other-repo-pipeline</id><content type="html" xml:base="https://lrita.github.io/2024/03/20/gitlab-trigger-other-repo-pipeline/"><![CDATA[<p>在一些项目中出现关联关系时，想在A项目触发B项目的<code class="language-plaintext highlighter-rouge">pipeline</code>时，可以参考的例子：</p>

<h2 id="项目b">项目B</h2>

<p>项目B为子项目，是被触发的一方。</p>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">stages</span><span class="pi">:</span>
  <span class="pi">-</span> <span class="s">stage_1</span>

<span class="na">xxx_job</span><span class="pi">:</span>
  <span class="na">stage</span><span class="pi">:</span> <span class="s">stage_1</span>
  <span class="na">rules</span><span class="pi">:</span>
    <span class="pi">-</span> <span class="na">if</span><span class="pi">:</span> <span class="s">$CI_PIPELINE_SOURCE == "pipeline"</span> <span class="c1"># 外部项目触发，此处表示该job可以被外部项目触发，其他值可以参考 https://docs.gitlab.com/ee/ci/variables/predefined_variables.html</span>
    <span class="pi">-</span> <span class="na">if</span><span class="pi">:</span> <span class="s">$CI_COMMIT_REF_NAME == "master"</span>   <span class="c1"># 代码合并入主分支时</span>
  <span class="na">script</span><span class="pi">:</span>
    <span class="pi">-</span> <span class="s">echo "abc"</span>                            <span class="c1"># 具体任务命令</span>
</code></pre></div></div>

<h2 id="项目a">项目A</h2>

<p>项目A为父项目，是发起的一方。父项目有两种配置方式：</p>

<h3 id="方式一">方式一</h3>

<p>在项目A的对应job中，调用项目B的对应API进行触发：</p>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">stages</span><span class="pi">:</span>
  <span class="pi">-</span> <span class="s">deploy</span>

<span class="na">xxx_job</span><span class="pi">:</span>
  <span class="na">stage</span><span class="pi">:</span> <span class="s">deploy</span>
  <span class="na">script</span><span class="pi">:</span>
    <span class="pi">-</span> <span class="s">curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=main "https://gitlab.example.com/api/v4/projects/9/trigger/pipeline"</span>
  <span class="na">rules</span><span class="pi">:</span>
    <span class="pi">-</span> <span class="na">if</span><span class="pi">:</span> <span class="s">$CI_COMMIT_TAG</span>
  <span class="na">environment</span><span class="pi">:</span> <span class="s">production</span>
</code></pre></div></div>

<h3 id="方式二">方式二</h3>

<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">stages</span><span class="pi">:</span>
  <span class="pi">-</span> <span class="s">deploy</span>

<span class="na">xxx_job</span><span class="pi">:</span>
  <span class="na">stage</span><span class="pi">:</span> <span class="s">deploy</span>
  <span class="na">trigger</span><span class="pi">:</span>                         <span class="c1"># 使用trigger字段，表示该job是用来触发子pipeline的</span>
    <span class="na">project</span><span class="pi">:</span> <span class="s">xx_group/project-b</span>    <span class="c1"># 项目B对应的名称</span>
    <span class="na">branch</span><span class="pi">:</span> <span class="s">master</span>                 <span class="c1"># 对应项目的代码分支</span>
                                   <span class="c1"># trigger的下的其他字段功能可以参考 https://docs.gitlab.com/ee/ci/yaml/index.html#trigger</span>
  <span class="na">rules</span><span class="pi">:</span>                           <span class="c1"># 可以用rules控制该job执行条件来控制触发条件</span>
    <span class="pi">-</span> <span class="na">if</span><span class="pi">:</span> <span class="s1">'</span><span class="s">$CI_MERGE_REQUEST_ID'</span>
</code></pre></div></div>

<p><img src="/images/posts/tools/gitlab-multi-repo-pipeline.png" alt="" /></p>

<h2 id="扩展">扩展</h2>

<p>其他更加复杂的用法可以参考<a href="https://docs.gitlab.com/ee/ci/pipelines/downstream_pipelines.html">文档</a>，gitlab还支持一些更加复杂的用法，比如项目A给项目B下发<code class="language-plaintext highlighter-rouge">pipeline</code>配置，转发环境变量等</p>]]></content><author><name>Neal Hu</name></author><category term="git" /><summary type="html"><![CDATA[git gitlab]]></summary></entry><entry><title type="html">发现项目中的重复代码</title><link href="https://lrita.github.io/2024/02/22/find-duplicated-code/" rel="alternate" type="text/html" title="发现项目中的重复代码" /><published>2024-02-22T00:00:00+08:00</published><updated>2024-02-22T00:00:00+08:00</updated><id>https://lrita.github.io/2024/02/22/find-duplicated-code</id><content type="html" xml:base="https://lrita.github.io/2024/02/22/find-duplicated-code/"><![CDATA[<h2 id="发现项目中的重复代码">发现项目中的重复代码</h2>

<p>当代码项目变大、参与的人增加时，就容易出现代码频繁复制，复用率降低的情况，倾向保守是人之常情。当问题积累到一定程度时，还是需要优化解决一下的。为了解决项目中的重复代码问题，我调研了多个工具，最终发现有两个工具还不错，简单易行，容易继承到CI中，特此介绍一下：</p>

<h2 id="simian">simian</h2>

<p><a href="https://simian.quandarypeak.com/">simian</a> 是一个专门用来检测重复代码的静态分析工具，基本上支持常见的各种编程语言，其使用起来也是非常简单。</p>

<p>第一步在<a href="https://simian.quandarypeak.com/download/">下载页面</a>下载工具包，备份<a href="/images/posts/tools/simian-academic.zip">simian-4.0.0.jar</a>，其依赖JDK17以上的Java环境，需要提前安装。</p>

<p>第二步解压压缩包，其中只有一个jar包。</p>

<p>第三步执行检测命令：</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># 参数 :</span>
<span class="c"># -threshold=10 设置检测重复代码行数阈值，超过指定行数时会被检测</span>
<span class="c"># -defaultLanguage=LANG 可选值：java, c#, cs, csharp, c, c++, cpp, cplusplus, js, \</span>
<span class="c">#                              javascript, cobol, abap, rb, ruby, vb, jsp, html, xml, \</span>
<span class="c">#                              groovy, asm390，检测不出源码对应语言时，假设为该种语言类型</span>
<span class="c"># -language=LANG 假设全部文件都为该种语言类型</span>
<span class="c"># -reportDuplicateText 显示重复的代码片段</span>
java <span class="nt">-jar</span> simian-4.0.0.jar <span class="nt">-language</span><span class="o">=</span>cpp <span class="nt">-reportDuplicateText</span> <span class="nt">-threshold</span><span class="o">=</span>10 ../../folly-v2023.03.13.00/<span class="k">**</span>/<span class="k">*</span>.h ../../folly-v2023.03.13.00/<span class="k">**</span>/<span class="k">*</span>.cpp
</code></pre></div></div>

<p>其会输出以下内容，可以从中看到重复的文件行数范围，以及重复的内容，可以选择合适的<code class="language-plaintext highlighter-rouge">-threshold</code>参数来进行扫描：</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>=====================================================================
Found 15 duplicate lines with fingerprint dfa3c0ad5fd93c0e3154c813486d85fa in the following files:
 Between lines 323 and 337 in /data1/neal/y/folly-v2023.03.13.00/folly/AtomicHashMap-inl.h
 Between lines 363 and 377 in /data1/neal/y/folly-v2023.03.13.00/folly/AtomicHashMap-inl.h
template &lt;
    typename KeyT,
    typename ValueT,
    typename HashFcn,
    typename EqualFcn,
    typename Allocator,
    typename ProbeFcn,
    typename KeyConvertFcn&gt;
typename AtomicHashMap&lt;
    KeyT,
    ValueT,
    HashFcn,
    EqualFcn,
    Allocator,
    ProbeFcn,
=====================================================================
</code></pre></div></div>

<p>其比较明显的缺点是，文件路径只能支持到固定格式层级的目录，比如上面”<em>*/</em>.h”这种，当源码文件夹较多，深度各自不同时，传递文件路径比较费事。</p>

<h2 id="pmd">pmd</h2>

<p><a href="https://pmd.github.io/pmd/index.html">pmd</a> 是一个开源静态分析工具，支持常见的16中变成语言，其除了检测重复代码外还有其他功能，此处只介绍其检测重复代码的CPD功能。</p>

<p>第一步，在<a href="https://github.com/pmd/pmd/releases">github</a>上下载最新发布二进制包<code class="language-plaintext highlighter-rouge">pmd-bin-xxx.zip</code>。</p>

<p>第二步解压压缩包，其中是一个完成的目录结构，不要破坏其原本的目录结构。</p>

<p>第三步执行检测命令：</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># --minimum-tokens 200 判断重复的最小token数，跟simian的-threshold比较类似，单位不是行，而是词组</span>
<span class="c"># --dir &lt;path&gt; / -d &lt;path&gt; 指定源码目录，比simian就灵活许多</span>
<span class="c"># --file-list &lt;filepath&gt;</span>
<span class="c"># --skip-duplicate-files 跳过重复的文件，这种在c的大规模项目比较常见</span>
<span class="c"># --skip-lexical-errors 这个比较需要，不要因为解析失败而中断检查</span>
./bin/pmd cpd <span class="nt">--language</span> cpp <span class="nt">--minimum-tokens</span> 200 <span class="nt">--skip-lexical-errors</span> <span class="nt">--dir</span> ../../folly-v2023.03.13.00
</code></pre></div></div>

<p>其会输出以下内容，可以从中看到重复的文件行数范围，以及重复的内容，可以选择合适的<code class="language-plaintext highlighter-rouge">--minimum-tokens</code>参数来进行扫描：</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>=====================================================================
Found 16 duplicate lines with fingerprint 916b4dfabbca319a79ecd278b6556e52 in the following files:
 Between lines 273 and 288 in /data1/neal/y/folly-v2023.03.13.00/folly/AtomicHashMap-inl.h
 Between lines 243 and 258 in /data1/neal/y/folly-v2023.03.13.00/folly/AtomicHashMap-inl.h
 Between lines 210 and 225 in /data1/neal/y/folly-v2023.03.13.00/folly/AtomicHashMap-inl.h
template &lt;
    typename KeyT,
    typename ValueT,
    typename HashFcn,
    typename EqualFcn,
    typename Allocator,
    typename ProbeFcn,
    typename KeyConvertFcn&gt;
template &lt;class LookupKeyT, class LookupHashFcn, class LookupEqualFcn&gt;
typename AtomicHashMap&lt;
    KeyT,
    ValueT,
    HashFcn,
    EqualFcn,
    Allocator,
    ProbeFcn,
=====================================================================
</code></pre></div></div>

<p>其输出的格式与<code class="language-plaintext highlighter-rouge">simian</code>比较类似，两个工具，从使用体感上，<code class="language-plaintext highlighter-rouge">pmd</code>略胜一筹，扫描发现的问题比较多一丢丢，比较推荐。</p>]]></content><author><name>Neal Hu</name></author><category term="tool" /><summary type="html"><![CDATA[tool]]></summary></entry><entry><title type="html">c++使用std::initializer_list构建常见多个连续或表达式的优雅写法</title><link href="https://lrita.github.io/2023/07/01/cpp-pertty-statement-style-with-initializer-list/" rel="alternate" type="text/html" title="c++使用std::initializer_list构建常见多个连续或表达式的优雅写法" /><published>2023-07-01T00:00:00+08:00</published><updated>2023-07-01T00:00:00+08:00</updated><id>https://lrita.github.io/2023/07/01/cpp-pertty-statement-style-with-initializer-list</id><content type="html" xml:base="https://lrita.github.io/2023/07/01/cpp-pertty-statement-style-with-initializer-list/"><![CDATA[<h2 id="问题一">问题一</h2>

<p>当代码逻辑中需要经常判断多个连续<code class="language-plaintext highlighter-rouge">||</code>表达式时，经常采用的模式都是：</p>
<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">if</span> <span class="p">(</span><span class="n">get_level</span><span class="p">()</span> <span class="o">==</span> <span class="s">"L1"</span> <span class="o">||</span> <span class="n">get_level</span><span class="p">()</span> <span class="o">==</span> <span class="s">"L2"</span> <span class="o">||</span> <span class="p">...</span> <span class="o">||</span> <span class="n">get_level</span><span class="p">()</span> <span class="o">==</span> <span class="s">"L10"</span><span class="p">)</span> <span class="p">{</span>
  <span class="p">...</span>
<span class="p">}</span>
</code></pre></div></div>
<p>当取值逻辑代价较高时，就需要减少取值的次数：</p>
<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">auto</span> <span class="n">level</span> <span class="o">=</span> <span class="n">get_level</span><span class="p">();</span>
<span class="k">if</span> <span class="p">(</span><span class="n">level</span> <span class="o">==</span> <span class="s">"L1"</span> <span class="o">||</span> <span class="n">level</span> <span class="o">==</span> <span class="s">"L2"</span> <span class="o">||</span> <span class="p">...</span> <span class="o">||</span> <span class="n">level</span> <span class="o">==</span> <span class="s">"L10"</span><span class="p">)</span> <span class="p">{</span>
  <span class="p">...</span>
<span class="p">}</span>
</code></pre></div></div>
<p>即使有C++17的加持，这种前置取值的写法经常也不是很优雅：</p>
<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">auto</span> <span class="n">level</span> <span class="o">=</span> <span class="n">get_level</span><span class="p">();</span>
<span class="k">auto</span> <span class="n">type</span>  <span class="o">=</span> <span class="n">get_type</span><span class="p">();</span>
<span class="k">if</span> <span class="p">(</span> <span class="p">(</span><span class="n">level</span> <span class="o">==</span> <span class="s">"L1"</span> <span class="o">||</span> <span class="n">level</span> <span class="o">==</span> <span class="s">"L2"</span> <span class="o">||</span> <span class="p">...</span> <span class="o">||</span> <span class="n">level</span> <span class="o">==</span> <span class="s">"L10"</span><span class="p">)</span> <span class="o">||</span> <span class="p">(</span><span class="n">type</span> <span class="o">==</span> <span class="n">TypeFirst</span> <span class="o">||</span> <span class="n">type</span> <span class="o">==</span> <span class="n">TypeSecond</span><span class="p">))</span> <span class="p">{</span>
  <span class="p">...</span>
<span class="p">}</span>
</code></pre></div></div>

<h2 id="问题二">问题二</h2>

<p>当需要写多个值<code class="language-plaintext highlighter-rouge">||</code>判断时，按上面的常规写法，还是显得啰嗦，因此有时，人们使用<code class="language-plaintext highlighter-rouge">std::unordered_set</code>来构建或值判断的集合：</p>
<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">static</span> <span class="n">std</span><span class="o">::</span><span class="n">unordered_set</span><span class="o">&lt;</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="o">&gt;</span> <span class="n">levels</span> <span class="o">=</span> <span class="p">{</span>
  <span class="s">"L1"</span><span class="p">,</span>
  <span class="s">"L2"</span><span class="p">,</span>
  <span class="p">...</span>
  <span class="s">"L10"</span><span class="p">,</span>
<span class="p">};</span>
<span class="k">static</span> <span class="n">std</span><span class="o">::</span><span class="n">unordered_set</span><span class="o">&lt;</span><span class="n">Type</span><span class="o">&gt;</span> <span class="n">types</span> <span class="o">=</span> <span class="p">{</span>
  <span class="n">TypeFirst</span><span class="p">,</span>
  <span class="n">TypeSecond</span><span class="p">,</span>
<span class="p">};</span>
<span class="k">if</span> <span class="p">(</span><span class="n">levels</span><span class="p">.</span><span class="n">contains</span><span class="p">(</span><span class="n">get_level</span><span class="p">())</span> <span class="o">||</span> <span class="n">types</span><span class="p">.</span><span class="n">contains</span><span class="p">(</span><span class="n">get_type</span><span class="p">()))</span> <span class="p">{</span>
  <span class="p">...</span>
<span class="p">}</span>
</code></pre></div></div>
<p>这样虽然解决了问题一，但是不满足零代价抽象，其实新增了额外的性能开销，<a href="https://godbolt.org/z/fr1qno5fq">例如</a>上面这个简单例子编译得到：</p>
<pre><code class="language-asm">        call    get_level[abi:cxx11]()    # 先获取get_level()
        mov     rsi, QWORD PTR [rsp+8]
        mov     rdi, QWORD PTR [rsp]
        mov     edx, 3339675911
        call    std::_Hash_bytes(void const*, unsigned long, unsigned long) # 计算返回值hash
        xor     edx, edx
        mov     edi, OFFSET FLAT:_ZL6levels
        mov     rcx, rax
        div     QWORD PTR _ZL6levels[rip+8]
        mov     rsi, rdx
        mov     rdx, rsp
        call    std::_Hashtable&lt;std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;, std::allocator&lt;std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; &gt;, std::__detail::_Identity, std::equal_to&lt;std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; &gt;, std::hash&lt;std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; &gt;, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits&lt;true, true, true&gt; &gt;::_M_find_before_node(unsigned long, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const&amp;, unsigned long) const
        test    rax, rax # hash find
        je      .L58
        cmp     QWORD PTR [rax], 0
        je      .L58
        ...
</code></pre>
<p>可见其需要最字符串计算hash，然后在进行hashmap的匹配，在数值集合不大的情况下，这种开销远高于几次简单的字符串比较。</p>

<h2 id="解决方案">解决方案</h2>

<p>基于上面的两种问题，现在总结出一个既满足零代价抽象，有写起来比较优雅的方式，使用一个方法<code class="language-plaintext highlighter-rouge">c_contains</code>：</p>
<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">#include</span> <span class="cpf">&lt;string_view&gt;</span><span class="cp">
#include</span> <span class="cpf">&lt;type_traits&gt;</span><span class="cp">
#include</span> <span class="cpf">&lt;initializer_list&gt;</span><span class="cp">
#include</span> <span class="cpf">&lt;algorithm&gt;</span><span class="cp">
</span>
<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">InputIt</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">T</span><span class="p">&gt;</span>
<span class="kt">bool</span> <span class="nf">c_contains</span><span class="p">(</span><span class="n">InputIt</span> <span class="n">begin</span><span class="p">,</span> <span class="n">InputIt</span> <span class="n">end</span><span class="p">,</span> <span class="n">T</span> <span class="o">&amp;&amp;</span><span class="n">value</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// (1)</span>
  <span class="k">return</span> <span class="n">std</span><span class="o">::</span><span class="n">find</span><span class="p">(</span><span class="n">begin</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;</span><span class="p">(</span><span class="n">value</span><span class="p">))</span> <span class="o">!=</span> <span class="n">end</span><span class="p">;</span>
<span class="p">}</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">C</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">initializer_list</span><span class="o">&lt;</span><span class="n">std</span><span class="o">::</span><span class="n">string_view</span><span class="p">&gt;,</span> <span class="k">typename</span> <span class="n">T</span><span class="o">&gt;</span>
<span class="kt">bool</span> <span class="n">c_contains</span><span class="p">(</span><span class="k">const</span> <span class="n">C</span> <span class="o">&amp;</span><span class="n">c</span><span class="p">,</span> <span class="n">T</span> <span class="o">&amp;&amp;</span><span class="n">value</span><span class="p">)</span> <span class="p">{</span>                                 <span class="c1">// (2)</span>
  <span class="k">return</span> <span class="n">c_contains</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">begin</span><span class="p">(</span><span class="n">c</span><span class="p">),</span> <span class="n">std</span><span class="o">::</span><span class="n">end</span><span class="p">(</span><span class="n">c</span><span class="p">),</span> <span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;</span><span class="p">(</span><span class="n">value</span><span class="p">));</span> <span class="c1">// 调用(1)</span>
<span class="p">}</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">U</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">T</span><span class="p">,</span>
          <span class="n">std</span><span class="o">::</span><span class="n">enable_if_t</span><span class="o">&lt;!</span><span class="n">std</span><span class="o">::</span><span class="n">is_same_v</span><span class="o">&lt;</span><span class="n">U</span><span class="p">,</span> <span class="k">const</span> <span class="kt">char</span> <span class="o">*</span><span class="p">&gt;,</span> <span class="kt">int</span> <span class="o">*&gt;</span> <span class="o">=</span> <span class="nb">nullptr</span><span class="o">&gt;</span>
<span class="kt">bool</span> <span class="n">c_contains</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">initializer_list</span><span class="o">&lt;</span><span class="n">U</span><span class="o">&gt;</span> <span class="n">c</span><span class="p">,</span> <span class="n">T</span> <span class="o">&amp;&amp;</span><span class="n">value</span><span class="p">)</span> <span class="p">{</span>
  <span class="k">return</span> <span class="n">c_contains</span><span class="p">(</span><span class="n">c</span><span class="p">.</span><span class="n">begin</span><span class="p">(),</span> <span class="n">c</span><span class="p">.</span><span class="n">end</span><span class="p">(),</span> <span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;</span><span class="p">(</span><span class="n">value</span><span class="p">));</span> <span class="c1">// 调用(2)</span>
<span class="p">}</span>
</code></pre></div></div>
<p>这里主要就是使用<code class="language-plaintext highlighter-rouge">std::initializer_list</code>和<code class="language-plaintext highlighter-rouge">std::find</code>来构成连续<code class="language-plaintext highlighter-rouge">||</code>值的判断：</p>
<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">int</span> <span class="nf">foo</span><span class="p">()</span> <span class="p">{</span>
  <span class="k">if</span> <span class="p">(</span><span class="n">c_contains</span><span class="p">({</span><span class="s">"L1"</span><span class="p">,</span> <span class="s">"L2"</span><span class="p">,</span> <span class="p">...</span> <span class="p">,</span> <span class="s">"L10"</span><span class="p">},</span> <span class="n">get_level</span><span class="p">())</span> <span class="o">||</span>
      <span class="n">c_contains</span><span class="p">({</span><span class="n">TypeFirst</span><span class="p">,</span> <span class="n">TypeSecond</span><span class="p">},</span> <span class="n">get_type</span><span class="p">()))</span> <span class="p">{</span>
    <span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
  <span class="p">}</span>
  <span class="k">return</span> <span class="mi">1</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>其<a href="https://godbolt.org/z/r4ePxYTdW">编译生成</a>的：</p>
<pre><code class="language-asm">        call    get_level[abi:cxx11]()
        mov     rbp, QWORD PTR [rsp+8]
        mov     r12, QWORD PTR [rsp]
        mov     eax, 2
        mov     edi, OFFSET FLAT:.LC0
        mov     ebx, OFFSET FLAT:._79
        jmp     .L15
.L2:
        cmp     QWORD PTR [rbx+16], rbp
        je      .L75
.L5:
        cmp     rbp, QWORD PTR [rbx+32]
        je      .L76
.L8:
        cmp     rbp, QWORD PTR [rbx+48]
        je      .L77
.L11:
        add     rbx, 64
        cmp     rbx, OFFSET FLAT:._79+128
        je      .L14
        mov     rdi, QWORD PTR [rbx+8]
        mov     rax, QWORD PTR [rbx]
.L15:
        cmp     rax, rbp
        jne     .L2
        test    rbp, rbp
        je      .L39
        mov     rdx, rbp
        mov     rsi, r12
        call    memcmp
        test    eax, eax
        jne     .L2
.L39:
        cmp     rbx, OFFSET FLAT:._79+160
        je      .L37
.L24:
        mov     rdi, r12
.L35:
        lea     rax, [rsp+16]
        xor     ebx, ebx
        cmp     rdi, rax
        je      .L1
.L33:
        call    operator delete(void*)
.L1:
        add     rsp, 32
        mov     eax, ebx
        pop     rbx
        pop     rbp
        pop     r12
        ret
.L77:
        test    rbp, rbp
        je      .L12
        mov     rdi, QWORD PTR [rbx+56]
        mov     rdx, rbp
        mov     rsi, r12
        call    memcmp
        test    eax, eax
        jne     .L11
.L12:
        add     rbx, 48
        jmp     .L39
.L76:
        test    rbp, rbp
        je      .L9
        mov     rdi, QWORD PTR [rbx+40]
        mov     rdx, rbp
        mov     rsi, r12
        call    memcmp
        test    eax, eax
        jne     .L8
.L9:
        add     rbx, 32
        jmp     .L39
.L75:
        test    rbp, rbp
        je      .L6
        mov     rdi, QWORD PTR [rbx+24]
        mov     rdx, rbp
        mov     rsi, r12
        call    memcmp
        test    eax, eax
        jne     .L5
.L6:
        add     rbx, 16
        jmp     .L39
.L14:
        cmp     rbp, 2
        je      .L78
        cmp     rbp, 3
        je      .L79
</code></pre>
<p>可以看出其直接转换成几句依次的<code class="language-plaintext highlighter-rouge">memcmp</code>，没有额外的抽象代价。</p>

<p>其中<code class="language-plaintext highlighter-rouge">c_contains(2)</code>是额外增加优化，发现在部分编译器上<code class="language-plaintext highlighter-rouge">{"L1", "L2", ... , "L10"}</code>被推断为<code class="language-plaintext highlighter-rouge">std::initializer_list&lt;const char *&gt;</code>，然后在后面<code class="language-plaintext highlighter-rouge">std::find</code>中，比较<code class="language-plaintext highlighter-rouge">const char *</code>和<code class="language-plaintext highlighter-rouge">std::string</code>存在抽象代价。因此使用<code class="language-plaintext highlighter-rouge">c_contains(2)</code>和<code class="language-plaintext highlighter-rouge">c_contains(3)</code>共同作用将<code class="language-plaintext highlighter-rouge">{"L1", "L2", ... , "L10"}</code>推断为<code class="language-plaintext highlighter-rouge">std::initializer_list&lt;std::string_view&gt;</code>，具体案例可以参考<a href="https://quick-bench.com/q/jVTCpwUHmtFhtD-gEKw4WP99kuQ">quick-bench</a>。</p>]]></content><author><name>Neal Hu</name></author><category term="c++" /><summary type="html"><![CDATA[c++]]></summary></entry><entry><title type="html">使用clang-tidy在CI中自动修复代码中简单问题和检测代码问题</title><link href="https://lrita.github.io/2023/03/21/auto-clang-tidy-cpp-code/" rel="alternate" type="text/html" title="使用clang-tidy在CI中自动修复代码中简单问题和检测代码问题" /><published>2023-03-21T00:00:00+08:00</published><updated>2023-03-21T00:00:00+08:00</updated><id>https://lrita.github.io/2023/03/21/auto-clang-tidy-cpp-code</id><content type="html" xml:base="https://lrita.github.io/2023/03/21/auto-clang-tidy-cpp-code/"><![CDATA[<p>一个几十人同时参与开发的C++项目，可以通过引入<code class="language-plaintext highlighter-rouge">clang-tidy</code>来帮助团队提升编码规范，可以讲题放到CI任务中，自动修复、检测一些常见问题，可以极大程度解放人力，并且缓解矛盾，<del>总有一些人瞎写、记性差，长久训练也无法提升，只靠code review也要在这些人身上消耗大量人力，而且这些人通常还会不停的ping你，企图用上线时间迫使你放松一些，赶紧把他那些不用心的代码合进去。用一个冷冰冰的CI也可以很方便的堵住这些人的嘴</del>。</p>

<p>下面是我基于<code class="language-plaintext highlighter-rouge">clang-tidy</code>的一些实践总结：</p>
<ul>
  <li>首先<code class="language-plaintext highlighter-rouge">clang-tidy</code>基本处于能用、可用的状态，不完美，bug还不少，但是也基本足够了。</li>
  <li><code class="language-plaintext highlighter-rouge">clang-tidy</code>能够检查很大一部分低幼问题</li>
  <li><code class="language-plaintext highlighter-rouge">clang-tidy</code>拥有自动修复模式(<code class="language-plaintext highlighter-rouge">-fix</code>)，但bug比较多，经常胡乱修复，下面案例中提供了一些我常用的配置项，这些项能够稳定自动修复。</li>
</ul>

<p>我们需要用到的工具有：</p>
<ul>
  <li><code class="language-plaintext highlighter-rouge">clang-tidy</code>二进制程序，编译安装<a href="https://github.com/llvm/llvm-project">llvm-project</a>就可以得到，单线程程序、每次检测一个源码文件；</li>
  <li><code class="language-plaintext highlighter-rouge">run-clang-tidy.py</code>脚本，貌似不会出现在llvm-project安装目录里，得从源码中拷贝出来。这个脚本主要并发执行N个<code class="language-plaintext highlighter-rouge">clang-tidy</code>进程；</li>
  <li><code class="language-plaintext highlighter-rouge">clang-tidy</code>的配置(<code class="language-plaintext highlighter-rouge">.ci/.clang-tidy</code>)、脚手架脚本等</li>
</ul>

<p>clang-tidy 配置</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>---
# 配置clang-tidy配置检测项，带'-'前缀的为disable对应的检测，否则为开启。这里主要是关闭一些用处不大，或者存在bug、假阳性的检查项
Checks: '*,
    -llvm-*,
    -llvmlibc-*,
    -altera-*,
    -android-*,
    -boost-*,
    -darwin-*,
    -fuchsia-*,
    -linuxkernel-*,
    -objc-*,
    -portability-*,
    -zircon-*,
    -clang-analyzer-osx*,
    -clang-analyzer-optin.cplusplus.UninitializedObject,
    -clang-analyzer-optin.cplusplus.VirtualCall,
    -clang-analyzer-core.NullDereference,
    -clang-analyzer-cplusplus.NewDelete,
    -clang-analyzer-cplusplus.PlacementNew,
    -clang-analyzer-cplusplus.NewDeleteLeaks,
    -clang-analyzer-cplusplus.Move,
    -clang-diagnostic-unused-parameter,
    -cppcoreguidelines-*,
    cppcoreguidelines-explicit-virtual-functions,
    cppcoreguidelines-special-member-functions,
    -cert-err58-cpp,
    -cert-env33-c,
    -cert-dcl37-c,
    -cert-dcl51-cpp,
    -google-runtime-int,
    -google-readability-casting,
    -google-readability-function-size,
    -google-readability-todo,
    -google-readability-braces-around-statements,
    -google-build-using-namespace,
    -readability-magic-numbers,
    -readability-implicit-bool-conversion,
    -readability-function-cognitive-complexity,
    -readability-isolate-declaration,
    -readability-convert-member-functions-to-static,
    -readability-container-size-empty,
    -readability-function-size,
    -readability-qualified-auto,
    -readability-make-member-function-const,
    -readability-named-parameter,
    -modernize-use-trailing-return-type,
    -modernize-avoid-c-arrays,
    -modernize-use-nullptr,
    -modernize-replace-disallow-copy-and-assign-macro,
    -modernize-use-bool-literals,
    -modernize-use-equals-default,
    -modernize-use-default-member-init,
    -modernize-use-auto,
    -modernize-loop-convert,
    -modernize-deprecated-headers,
    -modernize-raw-string-literal,
    -misc-no-recursion,
    -misc-unused-parameters,
    -misc-redundant-expression,
    -misc-non-private-member-variables-in-classes,
    -hicpp-*,
    hicpp-exception-baseclass,
    -performance-no-int-to-ptr,
    -bugprone-easily-swappable-parameters,
    -bugprone-implicit-widening-of-multiplication-result,
    -bugprone-integer-division,
    -bugprone-exception-escape,
    -bugprone-reserved-identifier,
    -bugprone-branch-clone,
    -bugprone-narrowing-conversions,
'
# 将警告转为错误
WarningsAsErrors: '*,-misc-non-private-member-variables-in-classes'
FormatStyle: file
# 过滤检查哪些头文件，clang-tidy会把源码依赖的头文件列出来都检查一遍，所以要屏蔽大量第三方库中的头文件
# 参考 https://stackoverflow.com/questions/71797349/is-it-possible-to-ignore-a-header-with-clang-tidy
# 该正则表达式引擎为llvm::Regex，支持的表达式较少，(?!xx)负向查找等都不支持
HeaderFilterRegex: '(xxx/include)*\.h$'
# 具体一些检查项的配置参数，可以参考的：
# https://github.com/envoyproxy/envoy/blob/main/.clang-tidy
# https://github.com/ClickHouse/ClickHouse/blob/d1d2f2c1a4979d17b7d58f591f56346bc79278f8/.clang-tidy
CheckOptions:
  - key: readability-identifier-naming.ClassCase
    value: CamelCase
  - key: readability-identifier-naming.EnumCase
    value: CamelCase
  - key: readability-identifier-naming.LocalVariableCase
    value: lower_case
  - key: readability-identifier-naming.StaticConstantCase
    value: aNy_CasE
  - key: readability-identifier-naming.PrivateMemberCase
    value: lower_case
  - key: readability-identifier-naming.PrivateMemberSuffix
    value: _
  - key: readability-identifier-naming.ProtectedMethodCase
    value: lower_case
  - key: readability-identifier-naming.ProtectedMethodSuffix
    value: _
  - key: readability-braces-around-statements.ShortStatementLines
    value: 2
  - key: readability-uppercase-literal-suffix.NewSuffixes
    value: 'f;u;ul'
  # Ignore GoogleTest function macros.
  - key: readability-identifier-naming.FunctionIgnoredRegexp
    value: '(TEST|TEST_F|TEST_P|INSTANTIATE_TEST_SUITE_P|MOCK_METHOD|TYPED_TEST)'
  - key: performance-move-const-arg.CheckTriviallyCopyableMove
    value: 0
  - key: cppcoreguidelines-special-member-functions.AllowSoleDefaultDtor
    value: 1
  - key: cppcoreguidelines-special-member-functions.AllowMissingMoveFunctions
    value: 1
  - key: cppcoreguidelines-special-member-functions.AllowMissingMoveFunctionsWhenCopyIsDeleted
    value: 1
</code></pre></div></div>

<p>封装脚本：封装脚本将<code class="language-plaintext highlighter-rouge">clang-tidy</code>的2个模式封装成了2个函数，可以在CI环境中依次调用两个函数，其中<code class="language-plaintext highlighter-rouge">auto_fix_simple_code</code>是修复模式，<code class="language-plaintext highlighter-rouge">clang_tidy_check_all</code>是检查模式。</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/bash</span>

<span class="k">function </span>say<span class="o">()</span> <span class="o">{</span>
  <span class="nb">echo</span> <span class="s2">"&gt;&gt; </span><span class="si">$(</span><span class="nb">date</span> <span class="s1">'+%Y-%m-%d %H:%M:%S'</span><span class="si">)</span><span class="s2"> </span><span class="nv">$*</span><span class="s2">"</span>
<span class="o">}</span>

<span class="k">function </span>cmd<span class="o">()</span> <span class="o">{</span>
  say <span class="s2">"@</span><span class="nv">$*</span><span class="s2">"</span>
  <span class="c"># shellcheck disable=SC2068</span>
  <span class="nv">$@</span> 2&gt;&amp;1
<span class="o">}</span>

<span class="k">function </span>join_by<span class="o">()</span> <span class="o">{</span>
  <span class="nb">local </span><span class="nv">IFS</span><span class="o">=</span><span class="s2">"</span><span class="nv">$1</span><span class="s2">"</span>
  <span class="nb">shift
  echo</span> <span class="s2">"</span><span class="nv">$*</span><span class="s2">"</span>
<span class="o">}</span>

<span class="k">function </span>auto_fix_simple_code<span class="o">()</span> <span class="o">{</span>
  <span class="c"># 可以被自动修复的检查项，下面是一些能够稳定修复的常见错误</span>
  <span class="nv">AUTO_FIX_CHECKS_CFG</span><span class="o">=(</span>
    <span class="s2">"-*"</span>
    <span class="s2">"modernize-use-nullptr"</span>
    <span class="s2">"modernize-use-override"</span>
    <span class="s2">"modernize-use-using"</span>
    <span class="s2">"modernize-make-shared"</span>
    <span class="s2">"boost-use-to-string"</span>
    <span class="s2">"readability-container-size-empty"</span>
    <span class="s2">"readability-redundant-access-specifiers"</span>
    <span class="s2">"readability-redundant-string-cstr"</span>
    <span class="s2">"readability-redundant-string-init"</span>
    <span class="s2">"readability-redundant-smartptr-get"</span>
    <span class="s2">"readability-redundant-control-flow"</span>
    <span class="s2">"google-readability-namespace-comments"</span>
    <span class="s2">"performance-unnecessary-copy-initialization"</span>
    <span class="s2">"performance-for-range-copy"</span>
    <span class="s2">"performance-noexcept-move-constructor"</span>
    <span class="s2">"clang-analyzer-deadcode.DeadStores"</span>
  <span class="o">)</span>
  <span class="nv">AUTO_FIX_CHECKS</span><span class="o">=</span><span class="si">$(</span>join_by <span class="s2">","</span> <span class="s2">"</span><span class="k">${</span><span class="nv">AUTO_FIX_CHECKS_CFG</span><span class="p">[@]</span><span class="k">}</span><span class="s2">"</span><span class="si">)</span>
  <span class="c">#</span>
  run-clang-tidy.py <span class="nt">-p</span> <span class="s2">"</span><span class="nv">$BUILD_DIRECTORY</span><span class="s2">"</span> <span class="se">\</span>
    <span class="nt">-checks</span><span class="o">=</span><span class="s2">"</span><span class="nv">$AUTO_FIX_CHECKS</span><span class="s2">"</span> <span class="se">\</span>
    <span class="nt">-fix</span> <span class="nv">$FILE</span> <span class="se">\</span>
    <span class="o">&gt;</span> /tmp/clang-tidy-fix.log 2&gt;&amp;1

  <span class="k">if</span> <span class="o">[[</span> <span class="nt">-n</span> <span class="s2">"</span><span class="k">${</span><span class="nv">GITLAB_CI</span><span class="k">}</span><span class="s2">"</span> <span class="o">&amp;&amp;</span> <span class="s2">"</span><span class="si">$(</span>git status <span class="nt">--short</span> | <span class="nb">wc</span> <span class="nt">-l</span><span class="si">)</span><span class="s2">"</span> <span class="o">!=</span> <span class="s2">"0"</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then
    </span><span class="nb">set</span> <span class="nt">-e</span> +o pipefail
    <span class="c"># 存在被自动修复的变更，提交修复变更代码</span>
    cmd git add <span class="nt">-u</span>
    cmd git commit <span class="nt">-m</span> <span class="s2">"自动修复常规问题"</span>
    cmd git push <span class="s2">"http://</span><span class="k">${</span><span class="nv">CI_USER</span><span class="k">}</span><span class="s2">:</span><span class="k">${</span><span class="nv">CI_PRIVATE_TOKEN</span><span class="k">}</span><span class="s2">@</span><span class="k">${</span><span class="nv">CI_REPOSITORY_URL</span><span class="p">#*@</span><span class="k">}</span><span class="s2">"</span> <span class="s2">"HEAD:</span><span class="k">${</span><span class="nv">CI_MERGE_REQUEST_SOURCE_BRANCH_NAME</span><span class="k">}</span><span class="s2">"</span>
    <span class="nb">exit </span>0
  <span class="k">fi</span>
<span class="o">}</span>

<span class="k">function </span>clang_tidy_check_all<span class="o">()</span> <span class="o">{</span>
  <span class="c"># 检查仍然存在的问题</span>
  say .ci/run-clang-tidy.py <span class="nt">-p</span><span class="o">=</span><span class="s2">"</span><span class="nv">$BUILD_DIRECTORY</span><span class="s2">"</span> <span class="se">\</span>
    <span class="nt">-config-file</span><span class="o">=</span><span class="s2">".ci/.clang-tidy"</span> <span class="nv">$FILE</span>
  .ci/run-clang-tidy.py <span class="nt">-p</span><span class="o">=</span><span class="s2">"</span><span class="nv">$BUILD_DIRECTORY</span><span class="s2">"</span> <span class="se">\</span>
    <span class="nt">-config-file</span><span class="o">=</span><span class="s2">".ci/.clang-tidy"</span> <span class="nv">$FILE</span> <span class="se">\</span>
    <span class="o">&gt;</span> /tmp/clang-tidy-issue.log 2&gt;&amp;1

  <span class="k">if</span> <span class="o">[[</span> <span class="nt">-n</span> <span class="s2">"</span><span class="k">${</span><span class="nv">GITLAB_CI</span><span class="k">}</span><span class="s2">"</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
    <span class="o">{</span>
      <span class="nb">echo</span> <span class="s2">"clang-tidy 检测结果："</span>
      <span class="nb">echo</span> <span class="s1">'```'</span>
      <span class="nb">grep</span> <span class="nt">-A</span> 2 <span class="nt">-E</span> <span class="s2">"error:.*</span><span class="se">\[</span><span class="s2">.*</span><span class="se">\]</span><span class="s2">"</span> /tmp/clang-tidy-issue.log
      <span class="nb">echo</span> <span class="s1">'```'</span>
      <span class="nb">echo</span> <span class="s2">"详情请点击pipeline⭕️图标进行查看"</span>
    <span class="o">}</span> <span class="o">&gt;</span> /tmp/clang-tidy-summary.log
    <span class="k">if</span> <span class="o">[[</span> <span class="si">$(</span><span class="nb">wc</span> <span class="nt">-l</span> &lt; <span class="s2">"/tmp/clang-tidy-summary.log"</span><span class="si">)</span> <span class="nt">-gt</span> 4 <span class="o">]]</span><span class="p">;</span> <span class="k">then
      </span>cmd add_comment <span class="s2">"@/tmp/clang-tidy-summary.log"</span> <span class="c"># add_comment 是CI中提供的一个命令，给对应MR中添加评论</span>
      <span class="nb">exit </span>255                                       <span class="c"># 使CI任务失败</span>
    <span class="k">fi
  else</span>
    <span class="o">{</span>
      <span class="nb">echo</span> <span class="s2">"clang-tidy 检测结果："</span>
      <span class="nb">echo</span> <span class="s1">'```'</span>
      <span class="nb">grep</span> <span class="nt">-E</span> <span class="s2">"error:.*</span><span class="se">\[</span><span class="s2">.*</span><span class="se">\]</span><span class="s2">"</span> /tmp/clang-tidy-issue.log | <span class="nb">grep</span> <span class="nt">-Eo</span> <span class="s2">"</span><span class="se">\[</span><span class="s2">.*</span><span class="se">\]</span><span class="s2">"</span> | <span class="nb">sort</span> | <span class="nb">uniq</span> <span class="nt">-c</span> | <span class="nb">sort</span> <span class="nt">-n</span>
      <span class="nb">echo</span> <span class="s1">'```'</span>
      <span class="nb">echo</span> <span class="s2">"详情请点击pipeline⭕️图标进行查看"</span>
    <span class="o">}</span> <span class="o">&gt;</span> /tmp/clang-tidy-summary.log
    <span class="nb">cat</span> /tmp/clang-tidy-issue.log
  <span class="k">fi</span>
<span class="o">}</span>

<span class="nv">BUILD_DIRECTORY</span><span class="o">=</span><span class="s2">"./build"</span>                  <span class="c"># cmake执行目录</span>
<span class="nv">SOURCE_DIRECTORY</span><span class="o">=</span><span class="k">${</span><span class="nv">CI_PROJECT_DIR</span><span class="k">:-</span><span class="si">$(</span><span class="nb">pwd</span><span class="si">)</span><span class="k">}</span> <span class="c"># 源码目录</span>
say <span class="s2">"build cmake in </span><span class="nv">$BUILD_DIRECTORY</span><span class="s2"> ..."</span>
<span class="nb">mkdir</span> <span class="nt">-p</span> <span class="nv">$BUILD_DIRECTORY</span>
<span class="c"># 执行cmake build，-DCMAKE_EXPORT_COMPILE_COMMANDS=ON 使cmake生成单文件编译依赖配置文件，后续clang-tidy执行需要依赖该配置</span>
<span class="c"># 会在cmake build目录下生成一个 compile_commands.jso n文件</span>
cmd <span class="nb">cd</span> <span class="s2">"</span><span class="nv">$BUILD_DIRECTORY</span><span class="s2">"</span> <span class="se">\</span>
  <span class="o">&amp;&amp;</span> cmd cmake <span class="s2">"</span><span class="nv">$SOURCE_DIRECTORY</span><span class="s2">"</span> <span class="nt">-DCMAKE_BUILD_TYPE</span>:STRING<span class="o">=</span>RelWithDebInfo <span class="nt">-DCMAKE_EXPORT_COMPILE_COMMANDS</span><span class="o">=</span>ON
cmd <span class="nb">cd</span> <span class="s2">"</span><span class="nv">$SOURCE_DIRECTORY</span><span class="s2">"</span>

<span class="k">if</span> <span class="o">[[</span> <span class="nt">-n</span> <span class="s2">"</span><span class="k">${</span><span class="nv">GITLAB_CI</span><span class="k">}</span><span class="s2">"</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span> <span class="c"># gitlab CI中会定义GITLAB_CI变量</span>
  <span class="c"># 运行在CI中</span>
  <span class="nv">M_SHA1</span><span class="o">=</span><span class="si">$(</span>git rev-parse origin/master<span class="si">)</span>
  <span class="c"># 过滤出本次MR中涉及修改的文件</span>
  <span class="nv">FILE</span><span class="o">=</span><span class="si">$(</span>git diff <span class="nt">--name-status</span> <span class="s2">"</span><span class="nv">$M_SHA1</span><span class="s2">"</span> | <span class="nb">grep</span> <span class="nt">-E</span> <span class="s2">"^(M|A)</span><span class="se">\s</span><span class="s2">+(include|src)/.*</span><span class="se">\.</span><span class="s2">(cc|cpp|h|hpp)$"</span> | <span class="nb">awk</span> <span class="s1">'!/tests/ { print $2 }'</span><span class="si">)</span>
  <span class="o">[[</span> <span class="s2">"</span><span class="nv">$FILE</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">""</span> <span class="o">]]</span> <span class="o">&amp;&amp;</span> <span class="nb">exit </span>0
<span class="k">else</span>
  <span class="c"># 手动执行</span>
  <span class="nv">FILE</span><span class="o">=</span><span class="s1">'.*\.(?:h|cc)*'</span>
<span class="k">fi

case</span> <span class="s2">"</span><span class="nv">$1</span><span class="s2">"</span> <span class="k">in
</span>fix<span class="p">)</span>
  auto_fix_simple_code
  <span class="p">;;</span>
check<span class="p">)</span>
  clang_tidy_check_all
  <span class="p">;;</span>
<span class="k">esac</span>
</code></pre></div></div>]]></content><author><name>Neal Hu</name></author><category term="c++" /><summary type="html"><![CDATA[c++]]></summary></entry><entry><title type="html">使用clang-format在CI中自动格式化C++代码</title><link href="https://lrita.github.io/2022/07/30/auto-clang-format-cpp-code/" rel="alternate" type="text/html" title="使用clang-format在CI中自动格式化C++代码" /><published>2022-07-30T00:00:00+08:00</published><updated>2022-07-30T00:00:00+08:00</updated><id>https://lrita.github.io/2022/07/30/auto-clang-format-cpp-code</id><content type="html" xml:base="https://lrita.github.io/2022/07/30/auto-clang-format-cpp-code/"><![CDATA[<p>一个几十人同时参与开发的C++项目，每周大约合并PR40-60个，人员水平参差不齐，<del>一些人写了好几年，连IDE都不会配置，完全当文本编辑器在用</del>，一些基本要求反复强调但是仍然很难达成一致。比如代码格式的问题，实在令人头疼。所以最好还是能够自动化解决比较好。</p>

<ul>
  <li>首先是确定使用<code class="language-plaintext highlighter-rouge">clang-format</code>来格式化代码，把格式上的设置写入配置文件<code class="language-plaintext highlighter-rouge">.clang-format</code>放在项目根目录上，然后安装<code class="language-plaintext highlighter-rouge">clang-format</code>和配合git的脚本<code class="language-plaintext highlighter-rouge">git-clang-format</code>。</li>
  <li>其次是每次只更新新增代码，避免比必要的更新历史代码记录</li>
  <li>然后就是如何只更新新增代码，其具体步骤如下</li>
</ul>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># 适用于macOS手动执行</span>
git co xx_squash
git reset <span class="nt">--soft</span> master
git clang-format <span class="nt">--style</span><span class="o">=</span>file
git co xx
git add <span class="nt">-u</span>
git commit <span class="nt">-m</span> <span class="s2">"xx"</span>
git push
git b <span class="nt">-D</span> xx_squash
</code></pre></div></div>

<p>在macOS和linux具体步骤还不一样，可能是2个环境安装的<code class="language-plaintext highlighter-rouge">git-clang-format</code>版本不同。以下命令可以写成一个脚本，放在CI里自动化执行。</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/bin/bash</span>
<span class="c"># 适用于linux</span>
<span class="nv">M_SHA1</span><span class="o">=</span><span class="si">$(</span>git rev-parse origin/master<span class="si">)</span>
git clang-format <span class="nt">--style</span> file <span class="s2">"</span><span class="nv">$M_SHA1</span><span class="s2">"</span>
<span class="k">if</span> <span class="o">[[</span> <span class="s2">"</span><span class="si">$(</span>git status <span class="nt">--short</span> | <span class="nb">wc</span> <span class="nt">-l</span><span class="si">)</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"0"</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then
  </span><span class="nb">exit </span>0
<span class="k">fi
</span>git add <span class="nt">-u</span>
git commit <span class="nt">-m</span> <span class="s2">"格式化代码"</span>
git push <span class="s2">"https://</span><span class="k">${</span><span class="nv">CI_PRIVATE_USER</span><span class="k">}</span><span class="s2">:</span><span class="k">${</span><span class="nv">CI_PRIVATE_TOKEN</span><span class="k">}</span><span class="s2">@</span><span class="k">${</span><span class="nv">CI_REPOSITORY_URL</span><span class="p">#*@</span><span class="k">}</span><span class="s2">"</span> <span class="s2">"HEAD:</span><span class="k">${</span><span class="nv">CI_MERGE_REQUEST_SOURCE_BRANCH_NAME</span><span class="k">}</span><span class="s2">"</span>
<span class="c"># 其中 CI_PRIVATE_USER，CI_PRIVATE_TOKEN 用于push代码的权限，可以在CI自定义变量中配置。</span>
</code></pre></div></div>]]></content><author><name>Neal Hu</name></author><category term="c++" /><summary type="html"><![CDATA[c++]]></summary></entry><entry><title type="html">每个程序员都应该了解的内存知识-3</title><link href="https://lrita.github.io/2022/06/20/programmer-should-know-about-memory-3/" rel="alternate" type="text/html" title="每个程序员都应该了解的内存知识-3" /><published>2022-06-20T00:00:00+08:00</published><updated>2022-06-20T00:00:00+08:00</updated><id>https://lrita.github.io/2022/06/20/programmer-should-know-about-memory-3</id><content type="html" xml:base="https://lrita.github.io/2022/06/20/programmer-should-know-about-memory-3/"><![CDATA[<h1 id="5-numa-支持">5 NUMA 支持</h1>

<p>在第2节中，我们看到，在一些机器上，访问特定物理内存区域的开销因访问来源的不同而不同。这类硬件需要操作系统和应用程序特别小心。我们将从NUMA硬件的一些细节开始，然后我们将介绍Linux内核为NUMA提供的一些支持。</p>

<h2 id="51-numa-硬件">5.1 NUMA 硬件</h2>

<p>非统一存储体系结构(non-uniform memory architectures)正变得越来越普遍。在最简单的NUMA形式中，处理器访问本地内存（见图2.3）比访问其他处理器的内存的开销更低。</p>

<p>NUMA还特别适用于大型机器。我们已经描述过了让多个处理器访问同一个内存的问题。对于商业硬件，所有处理器都将共享同一个北桥（暂时忽略AMD Opteron NUMA节点，它们有自己的问题）。这使得北桥成为一个严重的瓶颈，因为所有内存访问都是通过它路由的。当然，大型机器可以使用定制硬件代替北桥，但是，除非使用的内存芯片有多个端口，即它们可以从多个总线使用，否则仍然存在瓶颈。多端口RAM的构建和支持既复杂又昂贵，因此几乎从未使用过。</p>

<p>复杂性的下一步是AMD使用的模型，其中互连机制为未直接连接到RAM的处理器提供访问。除非想要任意增加直径（即任意两个节点之间的最大距离），否则以这种方式形成的结构的尺寸是有限的。</p>

<p><img src="/images/posts/cpu/cpumemory.20.png" alt="" />
图 5.1: Hypercubes</p>

<p>一种有效的节点拓扑结构是超立方体，它将节点数量限制在2<sup>C</sup>，其中C是每个节点的互连接口数量。对于所有具有2<sup>n</sup>CPU的系统，超立方体的直径最小。图5.1显示了前三个超立方体。每个超立方体的直径为C，绝对最小值。AMD的第一代Opteron处理器每个处理器有三个超级传输链路。至少有一个处理器必须在一条链路上连接一个南桥，这意味着，目前可以直接高效地实现C=2的超立方体。下一代被宣布有四个链接，在这一点上C=3超立方体是可能的。</p>

<p>然而，这并不意味着无法支持更大的处理器堆积。有一些公司已经开发了横杆，允许使用更大的处理器集合（例如，Newisys的Horus）。但这些交叉链接增加了NUMA因子，但是在一定数量以上的处理器上不再高效。</p>

<p>下一步是连接多组CPU，并为它们实现共享内存。这样的设计存在多层复杂性。IBM x445和类似的机器仍然是一个非常接近商业级机器的系统。它们被设计成有x86和x86-64处理器的普通4U、8路机器。这些机器中的两台（有时多达四台）可以连接起来，作为一台具有共享内存的机器工作。所使用的互连引入了一个重要的NUMA因素，操作系统和应用程序都必须考虑到这一因素。</p>

<p>另一方面，像SGI Altix这样的机器是专门为互联而设计的，SGI的NUMAlink互连结构速度非常快和延迟很低；这两项都是高性能计算（HPC）的要求，特别是在使用MPI时。当然，缺点是这种复杂和专业化设计有非常高的代价。它们使NUMA系数合理较低成为可能，但由于这些机器可以拥有的CPU数量（数千）和互连的容量有限，NUMA系数实际上是动态的，并且可以达到不可接受的水平，具体取决于工作负载。</p>

<p>更常用的解决方案是使用高速网络连接商业机器集群。但这些不是NUMA机器；它们不实现共享地址空间，因此不属于本文讨论的任何类别。</p>

<h2 id="52-支持numa的操作系统">5.2 支持NUMA的操作系统</h2>

<p>为了支持NUMA机器，操作系统必须考虑内存的分布式特性。例如，如果一个进程在给定的处理器上运行，分配给该进程地址空间的物理RAM应该来自本地内存，否则，每条指令都必须访问远程内存以获取代码和数据。但是有一些情况需要在NUMA机器上特殊考虑。DSO的文本段通常在机器的物理RAM中只出现一次，但是，如果所有CPU上的进程和线程都使用该DSO（例如，基本的运行时库，如libc），这意味着除少数处理器外，所有处理器都必须具有远程访问。理想情况下，操作系统会将此类DSO“镜像”到每个处理器的物理RAM中，并使用本地副本。这是一种优化，不是一种要求，通常很难实现。</p>

<p>为了避免让情况变得更糟，操作系统不应该将进程或线程从一个NUMA节点迁移到另一个NUMA节点。操作系统应该已经尝试避免在普通多处理器机器上迁移进程，因为从一个处理器迁移到另一个处理器意味着缓存内容丢失。如果负载分配需要迁移处理器的进程或线程，操作系统通常可以选择一个具有足够剩余容量的任意新处理器。在NUMA环境中，新处理器的选择有点限制。新选择的处理器对进程正在使用的内存的访问成本不应高于旧处理器；这限制了目标列表。如果没有符合该标准的可用处理器，操作系统就别无选择，只能迁移到内存访问代价更高的处理器。</p>

<p>在这种情况下，有两种可能的方式进行(优化)。首先，我们可以希望这种情况是暂时的，并且可以将流程迁移回更适合的处理器。或者，操作系统也可以将进程的访问的内存迁移到最新访问它的处理器上。这是一个相当昂贵的操作。可能需要复制大量内存，尽管不一定是一步完成的。当这种情况发生时，必须停止该过程，至少是短暂地停止，以便正确迁移对旧页面的修改。要使页面迁移高效、快速，还有一系列其他要求。简而言之，除非真的有必要，否则操作系统应该避免它。</p>

<p>通常，不能假设NUMA机器上的所有进程都使用相同数量的内存，(不能假设)随着进程在处理器之间的分布，内存使用也会均匀分布。事实上，除非机器上运行的应用程序非常具体（在HPC世界中很常见，但在其他不常见），否则内存使用将非常不平均。一些应用程序将使用大量内存，而其他应用程序几乎没有。如果总是将内存分配给发出请求的处理器，这迟早会导致问题。系统最终将耗尽NUMA节点的本地内存。</p>

<p>作为对这些严重问题的响应，默认情况下，内存不是仅在本地节点上分配的。要利用系统的所有内存，默认策略是将内存条带化。这保证了系统所有内存的平等使用。作为一个副作用，可以在处理器之间自由迁移进程，因为平均而言，对所使用的所有内存的访问成本不会改变。对于较小的NUMA系数，条带化是可以接受的，但仍然不是最佳的（见第5.4节中的数据）。</p>

<p>这是一种悲观情绪，有助于系统避免严重问题，并使其在正常运行时更具可预测性。但在某些情况下，它确实会显著降低系统的整体性能。这就是为什么Linux允许每个进程选择内存分配规则。一个进程可以为自己及其子进程选择不同的策略。我们将在第6节介绍可用于此目的的接口。</p>

<h2 id="53-公开的信息">5.3 公开的信息</h2>

<p>这个章节感觉没有意思。</p>
<h2 id="54-访问非本节点内存的开销">5.4 访问非本节点内存的开销</h2>

<p>不过，NUMA节点距离是相关的。在[amdccnuma]AMD文档记录了四插槽机器的NUMA开销。对于写操作，如图5.3所示。</p>

<p><img src="/images/posts/cpu/cpumemory.49.png" alt="" />
图 5.3: 多节点之上的读写性能</p>

<p>写比读慢，这并不奇怪。有趣的是单跳和两跳的开销。其中两个单跳场景的开销实际上略有不同。有关详细信息，请参见[amdccnuma]。从这个图表中我们需要记住的事实是，两跳读取和写入分别比0跳读取慢30%和49%。两跳写入比0跳写入慢32%，比单跳写入慢17%。处理器和内存节点的相对位置会产生很大的差异。AMD的下一代处理器将为每个处理器提供四条连贯的超级传输链路。在这种情况下，四CPU插座机器的直径为1（C=2）。对于八个CPU插座，同样的问题再次出现，因为有八个节点的超立方体的直径是3（C=3）。</p>

<p>所有这些信息都是可用的，但使用起来很麻烦。在第6.5节中，我们将看到一个接口，它有助于更轻松地访问和使用这些信息。</p>

<p>最后一些系统能够提供的信息是进程本身的状态。这些信息可以确定内存映射文件、写时拷贝（COW）页面和匿名内存如何分布在系统中的节点上。每个进程都有一个文件<code class="language-plaintext highlighter-rouge">/proc/${PID}/numa_maps</code>，其中PID是进程的ID，如图5.2所示。</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>00400000 default <span class="nv">file</span><span class="o">=</span>/bin/cat <span class="nv">mapped</span><span class="o">=</span>3 <span class="nv">N3</span><span class="o">=</span>3
00504000 default <span class="nv">file</span><span class="o">=</span>/bin/cat <span class="nv">anon</span><span class="o">=</span>1 <span class="nv">dirty</span><span class="o">=</span>1 <span class="nv">mapped</span><span class="o">=</span>2 <span class="nv">N3</span><span class="o">=</span>2
00506000 default heap <span class="nv">anon</span><span class="o">=</span>3 <span class="nv">dirty</span><span class="o">=</span>3 <span class="nv">active</span><span class="o">=</span>0 <span class="nv">N3</span><span class="o">=</span>3
38a9000000 default <span class="nv">file</span><span class="o">=</span>/lib64/ld-2.4.so <span class="nv">mapped</span><span class="o">=</span>22 <span class="nv">mapmax</span><span class="o">=</span>47 <span class="nv">N1</span><span class="o">=</span>22
38a9119000 default <span class="nv">file</span><span class="o">=</span>/lib64/ld-2.4.so <span class="nv">anon</span><span class="o">=</span>1 <span class="nv">dirty</span><span class="o">=</span>1 <span class="nv">N3</span><span class="o">=</span>1
38a911a000 default <span class="nv">file</span><span class="o">=</span>/lib64/ld-2.4.so <span class="nv">anon</span><span class="o">=</span>1 <span class="nv">dirty</span><span class="o">=</span>1 <span class="nv">N3</span><span class="o">=</span>1
38a9200000 default <span class="nv">file</span><span class="o">=</span>/lib64/libc-2.4.so <span class="nv">mapped</span><span class="o">=</span>53 <span class="nv">mapmax</span><span class="o">=</span>52 <span class="nv">N1</span><span class="o">=</span>51 <span class="nv">N2</span><span class="o">=</span>2
38a933f000 default <span class="nv">file</span><span class="o">=</span>/lib64/libc-2.4.so
38a943f000 default <span class="nv">file</span><span class="o">=</span>/lib64/libc-2.4.so <span class="nv">anon</span><span class="o">=</span>1 <span class="nv">dirty</span><span class="o">=</span>1 <span class="nv">mapped</span><span class="o">=</span>3 <span class="nv">mapmax</span><span class="o">=</span>32 <span class="nv">N1</span><span class="o">=</span>2 <span class="nv">N3</span><span class="o">=</span>1
38a9443000 default <span class="nv">file</span><span class="o">=</span>/lib64/libc-2.4.so <span class="nv">anon</span><span class="o">=</span>1 <span class="nv">dirty</span><span class="o">=</span>1 <span class="nv">N3</span><span class="o">=</span>1
38a9444000 default <span class="nv">anon</span><span class="o">=</span>4 <span class="nv">dirty</span><span class="o">=</span>4 <span class="nv">active</span><span class="o">=</span>0 <span class="nv">N3</span><span class="o">=</span>4
2b2bbcdce000 default <span class="nv">anon</span><span class="o">=</span>1 <span class="nv">dirty</span><span class="o">=</span>1 <span class="nv">N3</span><span class="o">=</span>1
2b2bbcde4000 default <span class="nv">anon</span><span class="o">=</span>2 <span class="nv">dirty</span><span class="o">=</span>2 <span class="nv">N3</span><span class="o">=</span>2
2b2bbcde6000 default <span class="nv">file</span><span class="o">=</span>/usr/lib/locale/locale-archive <span class="nv">mapped</span><span class="o">=</span>11 <span class="nv">mapmax</span><span class="o">=</span>8 <span class="nv">N0</span><span class="o">=</span>11
7fffedcc7000 default stack <span class="nv">anon</span><span class="o">=</span>2 <span class="nv">dirty</span><span class="o">=</span>2 <span class="nv">N3</span><span class="o">=</span>2
</code></pre></div></div>
<p>Figure 5.2: Content of /proc/PID/numa_maps</p>

<p>文件中的重要信息是N0到N3的值，它指示为节点0到3上的内存区域分配的内存页数。很可能该程序是在CPU核心被调度到节点3的上执行的。程序本身和脏页都分配在该NUMA节点上。只读映射，例如ld-2.4的第一个映射ld.so和libc-2.4.so。因此，共享文件区域设置归档文件也会分配到其他节点上。</p>

<p>如图5.3所示，对于单跳和双跳的读取，节点间的读取性能分别下降9%和30%。对于执行时，如果如果二级缓存miss，则需要调用这样的读取操作，则每个缓存线都会产生这些额外的开销。如果内存距离处理器较远，则超出缓存大小的大型工作负载的所有成本都必须增加9%/30%。</p>

<p><img src="/images/posts/cpu/cpumemory.66.png" alt="" />
Figure 5.4: Operating on Remote Memory</p>

<p>要查看真实世界中的效果，我们可以按照第3.5.1节测量带宽，但这一次，内存位于远端NUMA节点上，单跳的距离。与使用本地内存的数据相比，该测试的结果如图5.4所示。这些数字在两个方向上都有一些大的尖峰，这是测量多线程代码问题的结果，可以忽略。此图中的重要信息是，读取操作总是慢20%。这比图5.3中的9%要慢得多，图5.3中的9%很可能不是<code class="language-plaintext highlighter-rouge">uninterrupted</code>读/写操作的数字，可能指的是较旧的处理器版本。只有AMD知道。</p>

<p>对于适合缓存的工作集大小，写入和复制操作的性能也会降低20%。对于超过缓存大小的工作集，写入性能不会明显低于本地节点上的操作。互连的速度足够快，可以跟上内存的速度。主要因素是等待主内存的时间。</p>

<h1 id="参考">参考</h1>]]></content><author><name>Neal Hu</name></author><category term="linux" /><category term="memory" /><summary type="html"><![CDATA[linux memory]]></summary></entry><entry><title type="html">每个程序员都应该了解的内存知识-2</title><link href="https://lrita.github.io/2022/04/01/programmer-should-know-about-memory-2/" rel="alternate" type="text/html" title="每个程序员都应该了解的内存知识-2" /><published>2022-04-01T00:00:00+08:00</published><updated>2022-04-01T00:00:00+08:00</updated><id>https://lrita.github.io/2022/04/01/programmer-should-know-about-memory-2</id><content type="html" xml:base="https://lrita.github.io/2022/04/01/programmer-should-know-about-memory-2/"><![CDATA[<p>看这篇文章之前，可以先简单阅读一下<a href="https://zhuanlan.zhihu.com/p/105499858">《KVM 虚拟化详解》</a>，可以帮助理解以下的内容。</p>

<h2 id="4-虚拟内存virtual-memory">4 虚拟内存(Virtual Memory)<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup><sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup></h2>

<p>处理器的虚拟内存子系统为每个进程提供了虚拟地址空间的实现。这使每个进程都认为它在系统中是独立的。虚拟内存的优势在其他地方有详细描述，因此这里不再复述。相反，本节集中讨论虚拟内存子系统的实现细节和相关代价。</p>

<p>虚拟地址空间由CPU的内存管理单元（MMU）实现。操作系统必须填充页表数据结构（page table data structures），但大多数CPU自己完成其余工作。这实际上是一个相当复杂的机制；理解它的最佳方式是介绍用于描述虚拟地址空间的数据结构。</p>

<p>通常（程序）输入一个虚拟地址让MMU进行翻译（获得对应的物理地址）。通常对其价值几乎没有限制。虚拟地址在32位系统上是32-bit，在64位系统上是64-bit。在某些系统上，例如x86和x86-64，使用的地址实际上包含了另一个层次的间接寻址（内存分段）：这些体系结构使用的分段内存寻址，其会向每个逻辑地址添加一个偏移量。我们可以忽略这一部分，它是微不足道的，程序员不必关心内存处理的性能。{x86上的段限制与性能有关，但这是另一回事。}</p>

<h3 id="41-最简单的地址转换">4.1 最简单的地址转换</h3>

<p>有趣的部分是虚拟地址到物理地址的转换。MMU可以逐页重新映射地址。就像地址缓存排列的时候，虚拟地址被分割为不同的部分。这些部分被用来做多个表的索引，而这些表是被用来创建最终物理地址用的。对于最简单的模型，我们只有一级表。</p>

<p><img src="/images/posts/memory/cpumemory.18.png" alt="" /></p>

<p><code class="language-plaintext highlighter-rouge">图 4.1: 一层表结构的地址转换</code></p>

<p>图4.1显示了如何使用虚拟地址的不同部分。顶部用于选择页面目录中的条目；该目录中的每个条目都可以由操作系统单独设置。页面目录条目确定物理内存页面的地址；页面目录中的多个不同条目可以指向同一块物理地址。存储单元的完整物理地址是通过将页面目录中的页面地址与虚拟地址中的低地址相结合来确定的。页面目录条目还包含一些关于页面的附加信息，例如访问权限。</p>

<p>页面目录的数据结构存储在内存中。操作系统必须分配连续的物理内存，并将该内存区域的基址存储在一个特殊的寄存器中。然后，虚拟地址的适当位被用作页面目录的索引，页面目录实际上是一个目录项数组。</p>

<p>举个具体的例子，这是x86机器上4MB页面的布局。虚拟地址的偏移部分大小为22位，足以寻址4MB页面中的每个字节。虚拟地址的剩余10位选择页面目录中的1024个条目之一。每个条目都包含一个4MB页面的10位基址，该基址与偏移量结合形成一个完整的32位地址。</p>

<h3 id="42-多级页表">4.2 多级页表</h3>

<p>4MB页大小不是标准，它们会浪费大量内存，因为操作系统必须执行的许多操作都需要与内存页面对齐。对于4kB页面（32位机器上的标准，而且通常也是64位机器的标准），虚拟地址的偏移部分大小只有12bit，这将留下20bit作为页表目录的选择器。有2^20个条目的页表是不实用的。即使每个条目只有4个字节，表的大小也将是4MB。由于每个进程都可能有自己独立的页表目录，(那么)系统的大部分物理内存都会被这些页表目录占用。</p>

<p>解决方案是使用多级页表。然后它可以表示一个稀疏的巨大页表目录，在这个目录中，没有实际使用的区域不需要分配内存。因此，这种表示方式更加紧凑，使得在内存中有许多进程的页表而不会对性能造成太大影响。</p>

<p>如今，最复杂的页表结构包括四个层。图4.2显示了这种实现的示意图。</p>

<p><img src="/images/posts/memory/cpumemory.19-sm.png" alt="" />
<code class="language-plaintext highlighter-rouge">Figure 4.2: 4-层地址转换</code></p>

<p>在本例中，虚拟地址至少分为五个部分。其中四个部分是各种目录的索引。CPU使用专用寄存器指向第4级目录。4级到2级目录的内容是对下一级目录的引用。如果一个目录条目被标记为空，它显然不需要指向任何较低的目录。这样，页表树就可以稀疏而紧凑。与图4.1一样，1级目录的条目包括部分物理地址，以及访问权限等辅助数据。</p>

<p>为了确定与虚拟地址对应的物理地址，处理器首先确定最高级别目录的地址。该地址通常存储在寄存器中。然后，CPU获取与该目录对应的虚拟地址的索引部分，并使用该索引选择适当的条目。此条目是下一个目录的地址，该目录使用虚拟地址的下一部分编制索引。这个过程一直持续到到达1级目录，此时目录项的值是物理地址的较高部分。物理地址通过从虚拟地址添加页偏移位来完成。这个过程称为页表树遍历。一些处理器（如x86和x86-64）在硬件上执行此操作，其他处理器则需要操作系统的帮助。</p>

<p>系统上运行的每个进程可能都需要自己的页表树。可以部分共享树，但这是个例外。因此，如果页表树所需的内存尽可能小，就有利于性能和可伸缩性。理想的情况是将使用过的内存紧密地放在虚拟地址空间中；实际使用的物理地址(是否紧凑/连续)并不重要。一个小程序可能只需要在2级、3级和4级各使用一个目录，以及几个1级目录。在具有4kB页面和每个目录512个条目的x86-64上，这允许寻址2MB，总共有4个目录（每个级别一个目录）。1GB的连续内存可以通过一个目录寻址，用于级别2到4，512个目录用于级别1。</p>

<p>不过，假设所有内存都可以连续分配，那就太简单了。大多数情况下，出于灵活性的原因，进程的堆栈和堆区域被分配在地址空间几乎相反的一端。这使得任何一个区域都可以在需要时尽可能地增长。这意味着很可能需要两个级别2的目录，相应地，需要更多级别较低的目录。</p>

<p>但即便如此，这也并不总是符合当前的做法。出于安全原因，可执行文件的各个部分（代码、数据、堆、堆栈、DSO（又名共享库））映射到随机地址。随机化延伸到各个部分的相对位置；这意味着进程中使用的各种内存区域在整个虚拟地址空间中都很分布的很广(不会集中到某些地址范围内)。通过对随机地址的位数施加一些限制，可以限制范围，但在大多数情况下，它肯定不允许进程在2级和3级仅使用一个或两个目录运行。</p>

<p>如果性能真的比安全性重要得多，那么可以关闭随机化。操作系统通常至少会在虚拟内存中连续加载所有DSO。</p>

<h3 id="43-优化页表访问">4.3 优化页表访问</h3>

<p>页表的所有数据结构都保存在内存中。创建进程或更改页表时，会通知CPU。页表用于使用将每个虚拟地址解析为物理地址。更重要的是：在解析虚拟地址的过程中，每个级别至少使用一个目录。这需要最多四次内存访问（对于正在运行的进程的一次访问），这将导致速度很慢。可以将这些目录表条目视为普通数据，并将它们缓存在L1d、L2等中，但这仍然太慢。</p>

<p>从虚拟内存诞生之初，CPU设计师就采用了特殊的优化方法。一个简单的计算可以表明，只有将目录表条目保留在L1d和更高的缓存中，才会导致糟糕的性能。每个绝对地址计算都需要与页表深度对应的多个L1d访问。这些访问无法并行化，因为它们依赖于前一次查找的结果。在具有四级页表的机器上，仅此一项就至少需要12个周期。再加上L1d未命中的概率，结果是指令管道无法隐藏任何信息。额外的L1d访问还窃取了缓存的宝贵带宽。</p>

<p>因此，不只是缓存页表条目，而是缓存物理页地址的完整计算。出于代码和数据缓存工作的相同原因，这种缓存地址计算是有效的。由于虚拟地址的页偏移部分（页内偏移量）在物理页地址的计算中不起任何作用，因此只有虚拟地址的其余部分用作缓存的标记。根据页面大小，这意味着数百或数千条指令或数据对象共享同一标记，因此物理地址前缀相同。</p>

<p>存储计算值的高速缓存称为<code class="language-plaintext highlighter-rouge">Translation Look-Aside Buffer</code>（TLB）。它通常是一个小缓存，但是它必须非常快。与其他缓存一样，现代CPU提供多级TLB缓存；更高级别的缓存更大，速度也更慢。L1TLB的小尺寸通常通过使缓存与LRU逐出策略完全关联来弥补。</p>

<p>如上所述，用于访问TLB的标签(tag，即查找缓存的key)是虚拟地址的一部分。如果标记在缓存中有匹配项，则通过将虚拟地址的页偏移量添加到(读取到的)缓存值来(加合得到)最终的物理地址。这是一个非常快速的过程；必须这样做，因为CPU每条指令需要依赖于内存的物理地址，在某些情况下，还必须使用物理地址作为key进行缓存L2的查询。如果TLB查找未命中，处理器必须执行页表遍历；这可能相当昂贵。</p>

<p>如果地址在另一页上，通过软件或硬件预取代码或数据可能会隐式预取TLB的条目。这不能用于硬件预取，因为硬件可能会启动无效的页表遍历。因此，程序员不能依靠硬件预取来预取TLB条目。因此必须显示使用预取指令(如果需要的时候)。TLB就像数据和指令缓存一样，可以出现在多个级别。与数据缓存一样，TLB通常以两种形式出现：指令TLB（ITLB）和数据TLB（DTLB）。与其他缓存一样，L2TLB等更高级别的TLB通常是统一的。</p>

<h3 id="431-使用tlb的警告">4.3.1 使用TLB的警告</h3>

<p>TLB是处理器核心的全局资源。在处理器核心上执行的所有线程和进程都使用相同的TLB。由于虚拟地址到物理地址的转换取决于安装了哪个页表树，因此如果页表发生更改，CPU不能盲目地重用缓存的条目。每个进程都有不同的页表树（但不是同一进程中的线程），内核和VMM（虚拟机监控程序）也有不同的页表树（如果存在的话）。进程的地址空间布局也可能发生变化。有两种方法可以解决这个问题：</p>

<ul>
  <li>每当更改页表树时，TLB都会刷新。</li>
  <li>TLB条目的标记被扩展，以额外且唯一地标识它们所引用的页表树。</li>
</ul>

<p>在第一种情况下，只要执行上下文切换，就会刷新TLB。由于在大多数操作系统中，从一个线程/进程切换到另一个线程/进程需要执行一些内核代码，因此TLB刷新仅限于进入和离开内核地址空间。在虚拟化系统上，当内核调用VMM并返回时，也会发生这种情况(指TLB刷新)。</p>

<p>刷新TLB有效但代价高。例如，在执行系统调用时，内核代码可能会被限制为几千条指令，这些指令可能会涉及几个新page（或者一个hugepage，就像某些体系结构上的Linux那样）。这项工作只会在触摸页面时替换尽可能多的TLB条目。对于拥有128个ITLB和256个DTLB条目的Intel Core2体系结构，刷新TLB意味着不必要地刷新了100多个条目和200多个条目。当系统调用返回到同一进程时，所有被刷新的TLB条目都可能被再次使用，但它们将不在TLB了。内核或VMM中经常使用的代码也是如此。在进入内核的每个条目上，必须从头开始填充TLB，即使内核和VMM的页表通常不会更改，因此，理论上，TLB条目可以保留很长时间。这也解释了为什么当今处理器中的TLB缓存并不更大：程序很可能不会运行足够长的时间来填充所有这些条目。</p>

<p>当然，这一事实并没有逃过CPU架构师的眼睛。优化缓存刷新的一种可能性是独立分别使TLB条目无效。例如，如果内核代码和数据属于特定的地址范围，则只有属于该地址范围的页面必须从TLB中移出。这只需要比较标签，因此并不十分昂贵。如果地址空间的一部分发生更改，例如通过调用munmap，此方法也很有用。</p>

<p>更好的解决方案是扩展用于TLB访问的标签。如果除了虚拟地址的一部分之外，还为每个页表树（即进程的地址空间）添加了一个唯一标识符，那么TLB根本不需要完全刷新。内核、VMM和各个进程都可以有唯一的标识符。该方案的唯一问题是，TLB标签的可用位数受到严重限制，而地址空间的数量则不受限制。这意味着一些标识符重用是必要的。发生这种情况时，必须部分刷新TLB（如果可能的话）。所有具有重用标识符的条目都必须刷新，但希望这是一个小得多的集合。</p>

<p>当系统上运行多个进程时，这种扩展的TLB标记具有普遍优势。如果每个可运行进程的内存使用（以及TLB项使用）都是有限的，那么当进程再次被调度时，最近使用的TLB项很有可能仍在TLB中。但还有两个额外的优势：</p>

<ul>
  <li>特殊的地址空间，比如内核和VMM使用的地址空间，通常只进入使用很短的时间；之后，控制权通常会返回到发起调用的地址空间。如果没有标签(tag)，将执行两次TLB刷新。使用标记时，调用地址空间的缓存翻译会被保留，而且由于内核和VMM地址空间根本不经常更改TLB条目，因此以前的系统调用等的翻译仍然可以使用。</li>
  <li>在同一进程的两个线程之间切换时，根本不需要TLB刷新。如果没有扩展的TLB标签，进入内核切换时会破坏另一个线程的TLB。</li>
</ul>

<p>一段时间以来，一些处理器已经实现了这些扩展标记。AMD推出了带有Pacifica虚拟化扩展的1-bit标签扩展。在虚拟化环境中，此<code class="language-plaintext highlighter-rouge">1-bit Address Space ID（ASID）</code>用于区分VMM的地址空间和guest域的地址空间。这允许操作系统避免在每次输入VMM（例如，为了处理页面错误）时刷新guest的TLB条目，或者在控制返回给guest时刷新VMM的TLB条目。该体系结构将允许在未来使用更多比特。其他主流处理器可能也会效仿并支持这一功能。</p>

<h3 id="432-影响tlb性能的因素">4.3.2 影响TLB性能的因素</h3>

<p>有几个因素会影响TLB性能。首先是页面的大小。显然，一个页面越大，它容纳的指令或数据对象就越多。因此，较大的页面大小会减少所需的地址转换总数，这意味着TLB缓存中需要的条目更少。大多数架构允许使用多种不同的页面大小；有些尺寸可以同时使用。例如，x86/x86-64处理器的正常页面大小为4kB，但它们也可以分别使用4MB和2MB页面。IA-64和PowerPC允许64kB这样的大小作为基本页大小。</p>

<p>不过，使用大页面会带来一些问题。用于大页面的内存区域在物理内存中必须是连续的。如果管理物理内存的单元大小提高到虚拟内存页的大小，则浪费的内存量将增加。所有类型的内存操作（如加载可执行文件）都需要与页面边界对齐。这意味着，平均而言，每个映射在物理内存中浪费的页面大小是每个映射的一半。这种浪费很容易累积起来；因此，它为物理内存分配的合理单元大小设置了上限。</p>

<p>将单元大小增加到2MB以适应x86-64上的大页面肯定是不现实的。这个尺寸太大了。但这反过来意味着每个大页面必须由许多小页面组成。这些小页面在物理内存中必须是连续的。分配单位页面大小为4kB的2MB连续物理内存可能是一项挑战。它需要找到一个包含512个连续页面的空闲区域。在系统运行一段时间后，物理内存变得碎片化，这可能非常困难（或不可能）。</p>

<p>因此，在Linux上，有必要在系统启动时使用特殊的hugetlbfs文件系统预先分配这些大页面。保留固定数量的物理页面作为大型虚拟页面专用。这会限制可能并不总是被使用的资源。它也是一个有限的池；增加它通常意味着重新启动系统。尽管如此，在追求性能优越、资源充足，hugepage仍然是一种选择。数据库服务器就是一个例子。</p>

<p>增加最小虚拟页面大小（相对于可选的大页面）也有问题。内存映射操作（例如加载应用程序）必须符合这些页面大小。对于大多数架构来说，可执行文件的各个部分的位置具有固定的关系。如果页面大小的增加超出了构建可执行文件或DSO时考虑的范围，则无法执行加载操作。记住这个限制很重要。图4.3显示了如何确定ELF二进制文件的对齐要求。它被编码在ELF程序头中。</p>

<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>eu-readelf <span class="nt">-l</span> /bin/ls
Program Headers:
  Type   Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
...
  LOAD   0x000000 0x0000000000400000 0x0000000000400000 0x0132ac 0x0132ac R E 0x200000
  LOAD   0x0132b0 0x00000000006132b0 0x00000000006132b0 0x001a71 0x001a71 RW  0x200000
...
</code></pre></div></div>
<p>Figure 4.3: ELF Program 头部指示对齐要求</p>

<p>在本例中，一个x86-64二进制文件的值为0x200000=2097152=2MB，对应于处理器支持的最大页面大小。</p>

<p>使用较大的页面大小还有第二个效果：页表树的级别数减少。由于虚拟地址中与页偏移量相对应的部分增加，因此不需要通过页目录处理多少位。这意味着，在TLB miss的情况下，需要完成的工作量会减少。</p>

<p>除了使用较大的页面大小，还可以通过将同时使用的数据移动到较少的页面来减少所需的TLB条目数量。这类似于我们上面讨论的缓存使用的一些优化。只是现在，所需的对齐量很大。鉴于TLB条目的数量非常小，这可能是一个重要的优化。</p>

<p>4.4 虚拟化的影响</p>

<p>操作系统映像的虚拟化将变得越来越普遍；这意味着图片中添加了另一层内存处理。进程（基本上是监狱）或操作系统容器的虚拟化不属于这一类，因为只涉及一个操作系统。Xen或KVM等技术可以在有或没有处理器帮助的情况下执行独立的操作系统映像。在这些情况下，只有一个软件可以直接控制对物理内存的访问。</p>

<p><img src="/images/posts/memory/cpumemory.34.png" alt="" /></p>

<p>Figure 4.4: Xen虚拟化模型</p>

<p>在Xen的情况下（见图4.4），Xen VMM就是该软件，不过VMM本身并没有实现许多其他硬件控件。与其他早期系统（以及Xen VMM的第一个版本）上的VMM不同，内存和处理器之外的硬件由特权Dom0域控制。目前，这与非特权DomU内核基本相同，就内存处理而言，它们没有区别。这里很重要的一点是，VMM将物理内存分配给Dom0和DomU内核，这些内核本身实现了通常的内存处理，就像它们直接在处理器上运行一样。</p>

<p>为了实现虚拟化完成所需的域分离，Dom0和DomU内核中的内存处理不具有对物理内存的无限制访问。VMM不会通过分发单独的物理页并让guest操作系统处理寻址操作；这不会对错误或恶意的guest域提供任何保护。相反，VMM为每个guest域创建自己的页表树，并使用这些数据结构寻址内存。好处是可以控制对页表树的管理信息的访问。如果代码没有适当的权限，它将无法执行任何操作。</p>

<p>这种访问控制在Xen提供的虚拟化中得到利用，无论是使用准虚拟化还是硬件（也称为完全）虚拟化。guest域为每个进程构建页面表树的方式故意与准虚拟化和硬件虚拟化非常相似。每当guest操作系统修改其页表时，就会调用VMM。VMM然后使用guest域中更新的信息来更新自己的影子页表。这些（影子页表）是硬件实际使用的页面表。显然，这个过程非常昂贵：页面表树的每次修改都需要调用VMM。虽然没有虚拟化，对内存映射的更改并不高效，但现在它们变得更加代价高。</p>

<p>考虑到从guest操作系统到切换到VMM，再切换回本身的更改已经非常代价高，额外的成本可能非常大。这就是为什么处理器开始有附加功能来避免创建影子页表。这不仅是因为速度问题，而且还减少了VMM的内存消耗。英特尔有<code class="language-plaintext highlighter-rouge">Extended Page Tables（EPT）</code>，AMD有<code class="language-plaintext highlighter-rouge">Nested Page Tables（NPT）</code>技术。本质上，这两种技术都有客户操作系统生成虚拟物理地址(注意，不是虚拟地址)的页表，然后，必须使用的EPT/NPT树将这些虚拟物理地址进一步转换为实际的物理地址。这将允许以几乎与无虚拟化情况相同的速度处理内存。它还减少了VMM的内存使用，因为现在每个域只需要维护一个页表树（与进程相反）。</p>

<p>附加功能的地址转换步骤的结果也存储在TLB中。这意味着TLB不存储虚拟物理地址，而是存储查找完的结果。已经有人解释说，AMD的Pacifica扩展引入ASID是为了避免每次进入时TLB刷新。ASID的bit数在处理器扩展的初始版本中为1位；这足以区分VMM和guest操作系统。英特尔的则是虚拟处理器ID（VPID），它们的用途相同，只是数量更多。但是，VPID对于每个guest域都是固定的，因此它不能用于标记单独的进程，也不能避免该级别的TLB刷新。</p>

<p>每个地址空间修改所需的工作量大是虚拟化操作系统的一个问题。不过，基于VMM的虚拟化还存在另一个固有问题：没有办法实现两层内存处理。但内存处理很难（特别是在考虑NUMA等复杂因素时，请参见第5节）。使用单独VMM的Xen方法使得优化（甚至是良好的）处理变得困难，因为内存管理实现的所有复杂性，包括内存区域发现等“琐碎”事情，都必须在VMM中复制。操作系统拥有成熟且优化的实现；人们真的希望避免重复它们。</p>

<p><img src="/images/posts/memory/cpumemory.35.png" alt="" />
Figure 4.5: KVM虚拟化模型</p>

<p>这就是为什么总结VMM/Dom0模型是如此有吸引力的选择。图4.5显示了KVM Linux内核扩展如何解决这个问题。没有单独的VMM直接在硬件上运行并控制所有guest；相反，一个普通的Linux内核接管了这个功能。这意味着Linux内核中完整而复杂的内存处理功能用于管理(虚拟化)系统的内存。guest域与正常的用户级进程一起运行，创建者称之为“guest mode”。虚拟化（准虚拟化或完全虚拟化）由另一个用户级进程(KVM VMM)控制。这只是另一个使用内核实现的特殊KVM设备控制来guest的过程。</p>

<p>与Xen模型的单独VMM相比，该模型的好处在于，即使在使用guest操作系统时仍有两个内存转换机制在工作，但只需要一个实现，即Linux内核中的实现。无需在另一段代码（如Xen VMM）中复制相同的功能。这会导致更少的工作、更少的错误，并且可能会减少两个内存处理程序接触的摩擦，因为Linux客户机中的内存处理程序与运行在裸硬件上的外部Linux内核中的内存处理程序做出相同的假设。</p>

<p>总的来说，程序员必须意识到，使用虚拟化后，内存操作的成本至少比不使用虚拟化时要高。任何减少这项工作的优化都将在虚拟化环境中获得更大的回报。随着时间的推移，处理器设计师将通过EPT和NPT等技术越来越多地减少差异，但这种差异永远不会完全消失。</p>

<h1 id="参考">参考</h1>
<div class="footnotes" role="doc-endnotes">
  <ol>
    <li id="fn:1" role="doc-endnote">
      <p><a href="/images/posts/memory/What-Every-Programmer-Should-Know-About-Memory.pdf">What-Every-Programmer-Should-Know-About-Memory</a> <a href="#fnref:1" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:2" role="doc-endnote">
      <p><a href="https://lwn.net/Articles/253361/">Memory part 3: Virtual Memory</a> <a href="#fnref:2" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
  </ol>
</div>]]></content><author><name>Neal Hu</name></author><category term="linux" /><category term="memory" /><summary type="html"><![CDATA[linux memory]]></summary></entry><entry><title type="html">字符串解析通用方法</title><link href="https://lrita.github.io/2021/11/05/cpp-string-utils/" rel="alternate" type="text/html" title="字符串解析通用方法" /><published>2021-11-05T00:00:00+08:00</published><updated>2021-11-05T00:00:00+08:00</updated><id>https://lrita.github.io/2021/11/05/cpp-string-utils</id><content type="html" xml:base="https://lrita.github.io/2021/11/05/cpp-string-utils/"><![CDATA[<p>最近业务上频繁需要处理多种分隔符拼接成的各种数据，总结了一种抽象程度高、解释度高、使用简洁的通用解析方式。</p>

<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">struct</span> <span class="nc">Student</span> <span class="p">{</span>
  <span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">name</span><span class="p">;</span>
  <span class="kt">float</span>       <span class="n">score1</span><span class="p">;</span>
  <span class="kt">float</span>       <span class="n">score2</span><span class="p">;</span>
<span class="p">};</span>

<span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">f</span> <span class="o">=</span> <span class="s">"Tom@20@0.5@0.6|John@21@0.6@0.6|Jean@16@0.2@0.1"</span><span class="p">;</span>
<span class="n">std</span><span class="o">::</span><span class="n">map</span><span class="o">&lt;</span><span class="n">std</span><span class="o">::</span><span class="n">string</span><span class="p">,</span> <span class="n">Student</span><span class="o">&gt;</span> <span class="n">student_map</span><span class="p">;</span>
<span class="c1">// 简洁: 体现在类似的两层数据表达在调用解析的时候，只需要这么简单。</span>
<span class="c1">// 抽象程度高: 体现在解析规则的lambda表达式可以支持多种builtin类型的自由组合，用户可以自己根据字符串格式要求自行声明对应格式的lambda表达式</span>
<span class="c1">//           然后该会将对应字符串转换成要求的类型，然后触发lambda调用。</span>
<span class="c1">// 解释度高: 体现在通过lambda表达式的声明就表达的对应字符串的格式，可以不再进行额外的注释，例如下面的例子，每个字段的含义可以通过lambda表达式</span>
<span class="c1">//          的函数声明体现。</span>
<span class="n">SplitString2ValueVec</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="sc">'|'</span><span class="p">,</span> <span class="sc">'@'</span><span class="p">,</span> <span class="p">[</span><span class="o">&amp;</span><span class="p">](</span><span class="n">std</span><span class="o">::</span><span class="n">string</span> <span class="n">name</span><span class="p">,</span> <span class="kt">int</span> <span class="n">age</span><span class="p">,</span> <span class="kt">float</span> <span class="n">score1</span><span class="p">,</span> <span class="kt">float</span> <span class="n">score2</span><span class="p">)</span> <span class="p">{</span>
  <span class="k">if</span> <span class="p">(</span><span class="n">age</span> <span class="o">&gt;</span> <span class="mi">18</span><span class="p">)</span> <span class="p">{</span>
    <span class="n">student_map</span><span class="p">.</span><span class="n">insert</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">Student</span><span class="p">{</span><span class="n">name</span><span class="p">,</span> <span class="n">score1</span><span class="p">,</span> <span class="n">score2</span><span class="p">});</span>
  <span class="p">}</span>
<span class="p">})</span>
</code></pre></div></div>

<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">#pragma once
#include</span> <span class="cpf">&lt;tuple&gt;</span><span class="cp">
#include</span> <span class="cpf">&lt;string&gt;</span><span class="cp">
#include</span> <span class="cpf">&lt;vector&gt;</span><span class="cp">
#include</span> <span class="cpf">&lt;string.h&gt;</span><span class="cp">
#include</span> <span class="cpf">&lt;boost/callable_traits.hpp&gt;</span><span class="cp">
#include</span> <span class="cpf">&lt;boost/type_traits.hpp&gt;</span><span class="cp">
#include</span> <span class="cpf">&lt;boost/utility/string_view.hpp&gt;</span><span class="cp">
#include</span> <span class="cpf">&lt;folly/Traits.h&gt;</span><span class="cp">
#include</span> <span class="cpf">&lt;folly/functional/ApplyTuple.h&gt;</span><span class="cp">
#include</span> <span class="cpf">&lt;absl/utility/utility.h&gt;</span><span class="cp">
#include</span> <span class="cpf">&lt;absl/strings/str_split.h&gt;</span><span class="cp">
#include</span> <span class="cpf">&lt;absl/strings/numbers.h&gt;</span><span class="cp">
</span>
<span class="k">namespace</span> <span class="n">common</span> <span class="p">{</span>
<span class="k">namespace</span> <span class="n">common_detail</span> <span class="p">{</span>
<span class="n">FOLLY_CREATE_HAS_MEMBER_TYPE_TRAITS</span><span class="p">(</span><span class="n">has_key_type_traits</span><span class="p">,</span> <span class="n">key_type</span><span class="p">);</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">T</span><span class="p">,</span> <span class="k">typename</span> <span class="o">=</span> <span class="kt">void</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">Conv2Type</span><span class="p">;</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">T</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">Conv2Type</span><span class="o">&lt;</span><span class="n">T</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">enable_if_t</span><span class="o">&lt;</span><span class="n">std</span><span class="o">::</span><span class="n">is_convertible</span><span class="o">&lt;</span><span class="n">T</span><span class="p">,</span> <span class="n">absl</span><span class="o">::</span><span class="n">string_view</span><span class="o">&gt;::</span><span class="n">value</span><span class="o">&gt;&gt;</span> <span class="p">{</span>
  <span class="kt">bool</span> <span class="k">operator</span><span class="p">()(</span><span class="n">absl</span><span class="o">::</span><span class="n">string_view</span> <span class="n">m</span><span class="p">,</span> <span class="n">T</span> <span class="o">*</span><span class="n">v</span><span class="p">)</span> <span class="p">{</span>
    <span class="o">*</span><span class="n">v</span> <span class="o">=</span> <span class="n">T</span><span class="p">(</span><span class="n">m</span><span class="p">);</span>
    <span class="k">return</span> <span class="nb">true</span><span class="p">;</span>
  <span class="p">}</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">T</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">Conv2Type</span><span class="o">&lt;</span><span class="n">T</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">enable_if_t</span><span class="o">&lt;</span><span class="n">std</span><span class="o">::</span><span class="n">is_integral</span><span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;::</span><span class="n">value</span><span class="o">&gt;&gt;</span> <span class="p">{</span>
  <span class="kt">bool</span> <span class="k">operator</span><span class="p">()(</span><span class="n">absl</span><span class="o">::</span><span class="n">string_view</span> <span class="n">m</span><span class="p">,</span> <span class="n">T</span> <span class="o">*</span><span class="n">v</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="n">absl</span><span class="o">::</span><span class="n">SimpleAtoi</span><span class="p">(</span><span class="n">m</span><span class="p">,</span> <span class="n">v</span><span class="p">);</span> <span class="p">}</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">Conv2Type</span><span class="o">&lt;</span><span class="kt">double</span><span class="o">&gt;</span> <span class="p">{</span>
  <span class="kt">bool</span> <span class="k">operator</span><span class="p">()(</span><span class="n">absl</span><span class="o">::</span><span class="n">string_view</span> <span class="n">m</span><span class="p">,</span> <span class="kt">double</span> <span class="o">*</span><span class="n">v</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="n">absl</span><span class="o">::</span><span class="n">SimpleAtod</span><span class="p">(</span><span class="n">m</span><span class="p">,</span> <span class="n">v</span><span class="p">);</span> <span class="p">}</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">Conv2Type</span><span class="o">&lt;</span><span class="kt">float</span><span class="o">&gt;</span> <span class="p">{</span>
  <span class="kt">bool</span> <span class="k">operator</span><span class="p">()(</span><span class="n">absl</span><span class="o">::</span><span class="n">string_view</span> <span class="n">m</span><span class="p">,</span> <span class="kt">float</span> <span class="o">*</span><span class="n">v</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="n">absl</span><span class="o">::</span><span class="n">SimpleAtof</span><span class="p">(</span><span class="n">m</span><span class="p">,</span> <span class="n">v</span><span class="p">);</span> <span class="p">}</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">Conv2Type</span><span class="o">&lt;</span><span class="kt">bool</span><span class="o">&gt;</span> <span class="p">{</span>
  <span class="kt">bool</span> <span class="k">operator</span><span class="p">()(</span><span class="n">absl</span><span class="o">::</span><span class="n">string_view</span> <span class="n">m</span><span class="p">,</span> <span class="kt">bool</span> <span class="o">*</span><span class="n">v</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="n">absl</span><span class="o">::</span><span class="n">SimpleAtob</span><span class="p">(</span><span class="n">m</span><span class="p">,</span> <span class="n">v</span><span class="p">);</span> <span class="p">}</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="kt">size_t</span> <span class="n">N</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">Tuple</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">It</span><span class="p">,</span> <span class="kt">bool</span> <span class="n">IsEnd</span> <span class="o">=</span> <span class="p">(</span><span class="n">N</span> <span class="o">==</span> <span class="n">std</span><span class="o">::</span><span class="n">tuple_size</span><span class="o">&lt;</span><span class="n">Tuple</span><span class="p">&gt;</span><span class="o">::</span><span class="n">value</span><span class="p">)</span><span class="o">&gt;</span>
<span class="k">struct</span> <span class="nc">TupleAssign</span> <span class="p">{</span>
  <span class="k">static</span> <span class="kt">bool</span> <span class="n">assign</span><span class="p">(</span><span class="n">It</span> <span class="n">it</span><span class="p">,</span> <span class="n">It</span> <span class="n">end</span><span class="p">,</span> <span class="n">Tuple</span> <span class="o">&amp;</span><span class="n">v</span><span class="p">)</span> <span class="p">{</span>
    <span class="k">static_assert</span><span class="p">(</span><span class="n">N</span> <span class="o">&lt;</span> <span class="n">std</span><span class="o">::</span><span class="n">tuple_size</span><span class="o">&lt;</span><span class="n">Tuple</span><span class="o">&gt;::</span><span class="n">value</span><span class="p">,</span> <span class="s">"bad assign index"</span><span class="p">);</span>
    <span class="k">if</span> <span class="p">(</span><span class="n">it</span> <span class="o">==</span> <span class="n">end</span><span class="p">)</span> <span class="p">{</span>
      <span class="k">return</span> <span class="nb">false</span><span class="p">;</span>
    <span class="p">}</span>
    <span class="k">auto</span> <span class="n">vv</span> <span class="o">=</span> <span class="o">*</span><span class="n">it</span><span class="p">;</span>
    <span class="k">return</span> <span class="n">TupleAssign</span><span class="o">&lt;</span><span class="n">N</span> <span class="o">+</span> <span class="mi">1</span><span class="p">,</span> <span class="n">Tuple</span><span class="p">,</span> <span class="n">It</span><span class="o">&gt;::</span><span class="n">assign</span><span class="p">(</span><span class="o">++</span><span class="n">it</span><span class="p">,</span> <span class="n">end</span><span class="p">,</span> <span class="n">v</span><span class="p">)</span> <span class="o">&amp;&amp;</span>
           <span class="n">Conv2Type</span><span class="o">&lt;</span><span class="k">typename</span> <span class="n">std</span><span class="o">::</span><span class="n">tuple_element</span><span class="o">&lt;</span><span class="n">N</span><span class="p">,</span> <span class="n">Tuple</span><span class="o">&gt;::</span><span class="n">type</span><span class="o">&gt;</span><span class="p">()(</span><span class="n">vv</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">std</span><span class="o">::</span><span class="n">get</span><span class="o">&lt;</span><span class="n">N</span><span class="o">&gt;</span><span class="p">(</span><span class="n">v</span><span class="p">));</span>
  <span class="p">}</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="kt">size_t</span> <span class="n">N</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">Tuple</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">It</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">TupleAssign</span><span class="o">&lt;</span><span class="n">N</span><span class="p">,</span> <span class="n">Tuple</span><span class="p">,</span> <span class="n">It</span><span class="p">,</span> <span class="nb">true</span><span class="o">&gt;</span> <span class="p">{</span>
  <span class="k">static</span> <span class="kt">bool</span> <span class="n">assign</span><span class="p">(</span><span class="n">It</span><span class="p">,</span> <span class="n">It</span><span class="p">,</span> <span class="n">Tuple</span> <span class="o">&amp;</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nb">true</span><span class="p">;</span> <span class="p">}</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">Callback</span><span class="p">&gt;</span>
<span class="k">using</span> <span class="n">SplitString2ValueVecCallbackReturnType</span> <span class="o">=</span>
    <span class="n">std</span><span class="o">::</span><span class="n">enable_if</span><span class="o">&lt;</span><span class="n">std</span><span class="o">::</span><span class="n">is_same</span><span class="o">&lt;</span><span class="kt">void</span><span class="p">,</span> <span class="n">boost</span><span class="o">::</span><span class="n">callable_traits</span><span class="o">::</span><span class="n">return_type_t</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;&gt;::</span><span class="n">value</span> <span class="o">||</span>
                   <span class="n">std</span><span class="o">::</span><span class="n">is_same</span><span class="o">&lt;</span><span class="kt">bool</span><span class="p">,</span> <span class="n">boost</span><span class="o">::</span><span class="n">callable_traits</span><span class="o">::</span><span class="n">return_type_t</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;&gt;::</span><span class="n">value</span><span class="o">&gt;</span><span class="p">;</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">T</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">RemoveTupleElementReference</span><span class="p">;</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span><span class="o">...</span> <span class="nc">T</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">RemoveTupleElementReference</span><span class="o">&lt;</span><span class="n">std</span><span class="o">::</span><span class="n">tuple</span><span class="o">&lt;</span><span class="n">T</span><span class="p">...</span><span class="o">&gt;&gt;</span> <span class="p">{</span>
  <span class="k">using</span> <span class="n">type</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">tuple</span><span class="o">&lt;</span><span class="n">boost</span><span class="o">::</span><span class="n">remove_cv_ref_t</span><span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;</span><span class="p">...</span><span class="o">&gt;</span><span class="p">;</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">Callback</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">SplitString2ValueVecCallbackArgsType</span> <span class="p">{</span>
  <span class="k">using</span> <span class="n">type</span> <span class="o">=</span> <span class="k">typename</span> <span class="n">RemoveTupleElementReference</span><span class="o">&lt;</span><span class="n">boost</span><span class="o">::</span><span class="n">callable_traits</span><span class="o">::</span><span class="n">args_t</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;&gt;::</span><span class="n">type</span><span class="p">;</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">T</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">IsVec</span> <span class="o">:</span> <span class="n">std</span><span class="o">::</span><span class="n">false_type</span> <span class="p">{</span> <span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">T</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">IsVec</span><span class="o">&lt;</span><span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;&gt;</span> <span class="o">:</span> <span class="n">std</span><span class="o">::</span><span class="n">true_type</span> <span class="p">{</span> <span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">Callback</span><span class="p">&gt;</span>
<span class="k">using</span> <span class="n">IsCallbackArgsOnly1Vec</span> <span class="o">=</span> <span class="n">folly</span><span class="o">::</span><span class="n">Conjunction</span><span class="o">&lt;</span>
    <span class="n">folly</span><span class="o">::</span><span class="n">bool_constant</span><span class="o">&lt;</span><span class="n">std</span><span class="o">::</span><span class="n">tuple_size</span><span class="o">&lt;</span><span class="n">boost</span><span class="o">::</span><span class="n">callable_traits</span><span class="o">::</span><span class="n">args_t</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;&gt;::</span><span class="n">value</span> <span class="o">==</span> <span class="mi">1</span><span class="o">&gt;</span><span class="p">,</span>
    <span class="n">IsVec</span><span class="o">&lt;</span><span class="n">boost</span><span class="o">::</span><span class="n">remove_cv_ref_t</span><span class="o">&lt;</span>
        <span class="k">typename</span> <span class="n">std</span><span class="o">::</span><span class="n">tuple_element</span><span class="o">&lt;</span><span class="mi">0</span><span class="p">,</span> <span class="n">boost</span><span class="o">::</span><span class="n">callable_traits</span><span class="o">::</span><span class="n">args_t</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;&gt;::</span><span class="n">type</span><span class="o">&gt;&gt;&gt;</span><span class="p">;</span>

<span class="k">struct</span> <span class="nc">TupleHasReferenceImpl</span> <span class="p">{</span>
  <span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span><span class="o">...</span> <span class="nc">T</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">ReturnType</span> <span class="o">=</span> <span class="n">folly</span><span class="o">::</span><span class="n">Disjunction</span><span class="o">&lt;</span><span class="n">folly</span><span class="o">::</span><span class="n">Conjunction</span><span class="o">&lt;</span>
                               <span class="n">std</span><span class="o">::</span><span class="n">is_lvalue_reference</span><span class="o">&lt;</span><span class="n">T</span><span class="p">&gt;,</span>
                               <span class="n">folly</span><span class="o">::</span><span class="n">Negation</span><span class="o">&lt;</span><span class="n">std</span><span class="o">::</span><span class="n">is_const</span><span class="o">&lt;</span><span class="n">std</span><span class="o">::</span><span class="n">remove_reference_t</span><span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;&gt;&gt;&gt;</span><span class="p">...</span><span class="o">&gt;&gt;</span>
  <span class="k">static</span> <span class="n">ReturnType</span> <span class="n">apply</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">tuple</span><span class="o">&lt;</span><span class="n">T</span><span class="p">...</span><span class="o">&gt;</span> <span class="o">&amp;&amp;</span><span class="p">)</span> <span class="p">{</span>
    <span class="k">return</span> <span class="p">{};</span>
  <span class="p">}</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">Tuple</span><span class="p">&gt;</span>
<span class="k">using</span> <span class="n">TupleHasReference</span> <span class="o">=</span> <span class="k">decltype</span><span class="p">(</span><span class="n">TupleHasReferenceImpl</span><span class="o">::</span><span class="n">apply</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">declval</span><span class="o">&lt;</span><span class="n">Tuple</span><span class="o">&gt;</span><span class="p">()));</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">Callback</span><span class="p">,</span>
          <span class="kt">bool</span> <span class="n">HasReference</span> <span class="o">=</span> <span class="n">TupleHasReference</span><span class="o">&lt;</span><span class="n">boost</span><span class="o">::</span><span class="n">callable_traits</span><span class="o">::</span><span class="n">args_t</span><span class="o">&lt;</span><span class="n">Callback</span><span class="p">&gt;</span><span class="o">&gt;::</span><span class="n">value</span><span class="o">&gt;</span>
<span class="k">struct</span> <span class="nc">ApplyInvoke</span> <span class="p">{</span>
 <span class="nl">public:</span>
  <span class="k">constexpr</span> <span class="k">decltype</span><span class="p">(</span><span class="k">auto</span><span class="p">)</span>
      <span class="k">operator</span><span class="p">()(</span><span class="n">Callback</span> <span class="o">&amp;&amp;</span>                                                     <span class="n">cb</span><span class="p">,</span>
                 <span class="k">typename</span> <span class="n">SplitString2ValueVecCallbackArgsType</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;::</span><span class="n">type</span> <span class="o">&amp;&amp;</span><span class="n">tuple</span><span class="p">)</span> <span class="k">const</span> <span class="p">{</span>
    <span class="k">return</span> <span class="n">folly</span><span class="o">::</span><span class="n">apply</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;</span><span class="p">(</span><span class="n">cb</span><span class="p">),</span> <span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">tuple</span><span class="p">));</span>
  <span class="p">}</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">Callback</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">ApplyInvoke</span><span class="o">&lt;</span><span class="n">Callback</span><span class="p">,</span> <span class="nb">true</span><span class="o">&gt;</span> <span class="p">{</span>
 <span class="nl">public:</span>
  <span class="k">constexpr</span> <span class="k">decltype</span><span class="p">(</span><span class="k">auto</span><span class="p">)</span>
      <span class="k">operator</span><span class="p">()(</span><span class="n">Callback</span> <span class="o">&amp;&amp;</span>                                                     <span class="n">cb</span><span class="p">,</span>
                 <span class="k">typename</span> <span class="n">SplitString2ValueVecCallbackArgsType</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;::</span><span class="n">type</span> <span class="o">&amp;&amp;</span><span class="n">tuple</span><span class="p">)</span> <span class="k">const</span> <span class="p">{</span>
    <span class="k">auto</span> <span class="n">tuple2</span> <span class="o">=</span> <span class="n">folly</span><span class="o">::</span><span class="n">forward_tuple</span><span class="p">(</span><span class="n">tuple</span><span class="p">);</span>
    <span class="k">return</span> <span class="n">folly</span><span class="o">::</span><span class="n">apply</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;</span><span class="p">(</span><span class="n">cb</span><span class="p">),</span> <span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">tuple2</span><span class="p">));</span>
  <span class="p">}</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">Callback</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">STRING</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">IsSupport</span> <span class="o">=</span> <span class="kt">void</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">SplitString2ValueVec</span><span class="p">;</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">Callback</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">STRING</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">SplitString2ValueVec</span><span class="o">&lt;</span>
    <span class="n">Callback</span><span class="p">,</span> <span class="n">STRING</span><span class="p">,</span>
    <span class="k">typename</span> <span class="n">std</span><span class="o">::</span><span class="n">enable_if</span><span class="o">&lt;</span>
        <span class="o">!</span><span class="n">IsCallbackArgsOnly1Vec</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;::</span><span class="n">value</span> <span class="o">&amp;&amp;</span>
        <span class="n">std</span><span class="o">::</span><span class="n">is_same</span><span class="o">&lt;</span><span class="kt">void</span><span class="p">,</span> <span class="n">boost</span><span class="o">::</span><span class="n">callable_traits</span><span class="o">::</span><span class="n">return_type_t</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;&gt;::</span><span class="n">value</span><span class="o">&gt;::</span><span class="n">type</span><span class="o">&gt;</span> <span class="p">{</span>
  <span class="c1">// 返回值为void类型的回调函数时:</span>
  <span class="kt">void</span> <span class="k">operator</span><span class="p">()(</span><span class="k">const</span> <span class="n">STRING</span> <span class="o">&amp;</span><span class="n">v</span><span class="p">,</span> <span class="kt">char</span> <span class="n">key_value_pair_delimiter</span><span class="p">,</span> <span class="kt">char</span> <span class="n">key_value_delimiter</span><span class="p">,</span>
                  <span class="n">Callback</span> <span class="o">&amp;&amp;</span><span class="n">cb</span><span class="p">)</span> <span class="p">{</span>
    <span class="k">for</span> <span class="p">(</span><span class="k">const</span> <span class="k">auto</span> <span class="o">&amp;</span><span class="n">kv_token</span> <span class="o">:</span> <span class="n">absl</span><span class="o">::</span><span class="n">StrSplit</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">key_value_pair_delimiter</span><span class="p">,</span> <span class="n">absl</span><span class="o">::</span><span class="n">SkipEmpty</span><span class="p">()))</span> <span class="p">{</span>
      <span class="k">typename</span> <span class="n">SplitString2ValueVecCallbackArgsType</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;::</span><span class="n">type</span> <span class="n">tuple</span><span class="p">;</span>
      <span class="k">auto</span> <span class="n">sp</span> <span class="o">=</span> <span class="n">absl</span><span class="o">::</span><span class="n">StrSplit</span><span class="p">(</span><span class="n">kv_token</span><span class="p">,</span> <span class="n">key_value_delimiter</span><span class="p">);</span>
      <span class="k">if</span> <span class="p">(</span><span class="n">TupleAssign</span><span class="o">&lt;</span><span class="mi">0</span><span class="p">,</span> <span class="k">decltype</span><span class="p">(</span><span class="n">tuple</span><span class="p">),</span> <span class="k">typename</span> <span class="k">decltype</span><span class="p">(</span><span class="n">sp</span><span class="p">)</span><span class="o">::</span><span class="n">const_iterator</span><span class="o">&gt;::</span><span class="n">assign</span><span class="p">(</span>
              <span class="n">sp</span><span class="p">.</span><span class="n">begin</span><span class="p">(),</span> <span class="n">sp</span><span class="p">.</span><span class="n">end</span><span class="p">(),</span> <span class="n">tuple</span><span class="p">))</span> <span class="p">{</span>
        <span class="n">ApplyInvoke</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;</span> <span class="p">{}(</span><span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;</span><span class="p">(</span><span class="n">cb</span><span class="p">),</span> <span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">tuple</span><span class="p">));</span>
      <span class="p">}</span>
    <span class="p">}</span>
  <span class="p">}</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">Callback</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">STRING</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">SplitString2ValueVec</span><span class="o">&lt;</span>
    <span class="n">Callback</span><span class="p">,</span> <span class="n">STRING</span><span class="p">,</span>
    <span class="k">typename</span> <span class="n">std</span><span class="o">::</span><span class="n">enable_if</span><span class="o">&lt;</span>
        <span class="o">!</span><span class="n">IsCallbackArgsOnly1Vec</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;::</span><span class="n">value</span> <span class="o">&amp;&amp;</span>
        <span class="n">std</span><span class="o">::</span><span class="n">is_same</span><span class="o">&lt;</span><span class="kt">bool</span><span class="p">,</span> <span class="n">boost</span><span class="o">::</span><span class="n">callable_traits</span><span class="o">::</span><span class="n">return_type_t</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;&gt;::</span><span class="n">value</span><span class="o">&gt;::</span><span class="n">type</span><span class="o">&gt;</span> <span class="p">{</span>
  <span class="c1">// 返回值为bool类型的回调函数时，返回值为false时，中断解析。</span>
  <span class="kt">void</span> <span class="k">operator</span><span class="p">()(</span><span class="k">const</span> <span class="n">STRING</span> <span class="o">&amp;</span><span class="n">v</span><span class="p">,</span> <span class="kt">char</span> <span class="n">key_value_pair_delimiter</span><span class="p">,</span> <span class="kt">char</span> <span class="n">key_value_delimiter</span><span class="p">,</span>
                  <span class="n">Callback</span> <span class="o">&amp;&amp;</span><span class="n">cb</span><span class="p">)</span> <span class="p">{</span>
    <span class="k">for</span> <span class="p">(</span><span class="k">const</span> <span class="k">auto</span> <span class="o">&amp;</span><span class="n">kv_token</span> <span class="o">:</span> <span class="n">absl</span><span class="o">::</span><span class="n">StrSplit</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">key_value_pair_delimiter</span><span class="p">,</span> <span class="n">absl</span><span class="o">::</span><span class="n">SkipEmpty</span><span class="p">()))</span> <span class="p">{</span>
      <span class="k">typename</span> <span class="n">SplitString2ValueVecCallbackArgsType</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;::</span><span class="n">type</span> <span class="n">tuple</span><span class="p">;</span>
      <span class="k">auto</span> <span class="n">sp</span> <span class="o">=</span> <span class="n">absl</span><span class="o">::</span><span class="n">StrSplit</span><span class="p">(</span><span class="n">kv_token</span><span class="p">,</span> <span class="n">key_value_delimiter</span><span class="p">);</span>
      <span class="k">if</span> <span class="p">(</span><span class="n">TupleAssign</span><span class="o">&lt;</span><span class="mi">0</span><span class="p">,</span> <span class="k">decltype</span><span class="p">(</span><span class="n">tuple</span><span class="p">),</span> <span class="k">typename</span> <span class="k">decltype</span><span class="p">(</span><span class="n">sp</span><span class="p">)</span><span class="o">::</span><span class="n">const_iterator</span><span class="o">&gt;::</span><span class="n">assign</span><span class="p">(</span>
              <span class="n">sp</span><span class="p">.</span><span class="n">begin</span><span class="p">(),</span> <span class="n">sp</span><span class="p">.</span><span class="n">end</span><span class="p">(),</span> <span class="n">tuple</span><span class="p">))</span> <span class="p">{</span>
        <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">ApplyInvoke</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;</span> <span class="p">{}(</span><span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;</span><span class="p">(</span><span class="n">cb</span><span class="p">),</span> <span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">tuple</span><span class="p">)))</span> <span class="p">{</span>
          <span class="k">break</span><span class="p">;</span>
        <span class="p">}</span>
      <span class="p">}</span>
    <span class="p">}</span>
  <span class="p">}</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">Callback</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">STRING</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">SplitString2ValueVec</span><span class="o">&lt;</span>
    <span class="n">Callback</span><span class="p">,</span> <span class="n">STRING</span><span class="p">,</span>
    <span class="k">typename</span> <span class="n">std</span><span class="o">::</span><span class="n">enable_if</span><span class="o">&lt;</span>
        <span class="n">IsCallbackArgsOnly1Vec</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;::</span><span class="n">value</span> <span class="o">&amp;&amp;</span> <span class="c1">// 参数为std::vector&lt;T&gt;时</span>
        <span class="n">std</span><span class="o">::</span><span class="n">is_same</span><span class="o">&lt;</span><span class="kt">void</span><span class="p">,</span> <span class="n">boost</span><span class="o">::</span><span class="n">callable_traits</span><span class="o">::</span><span class="n">return_type_t</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;&gt;::</span><span class="n">value</span><span class="o">&gt;::</span><span class="n">type</span><span class="o">&gt;</span> <span class="p">{</span>
  <span class="c1">// 返回值为void类型的回调函数时:</span>
  <span class="kt">void</span> <span class="k">operator</span><span class="p">()(</span><span class="k">const</span> <span class="n">STRING</span> <span class="o">&amp;</span><span class="n">v</span><span class="p">,</span> <span class="kt">char</span> <span class="n">key_value_pair_delimiter</span><span class="p">,</span> <span class="kt">char</span> <span class="n">key_value_delimiter</span><span class="p">,</span>
                  <span class="n">Callback</span> <span class="o">&amp;&amp;</span><span class="n">cb</span><span class="p">)</span> <span class="p">{</span>
    <span class="k">for</span> <span class="p">(</span><span class="k">const</span> <span class="k">auto</span> <span class="o">&amp;</span><span class="n">kv_token</span> <span class="o">:</span> <span class="n">absl</span><span class="o">::</span><span class="n">StrSplit</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">key_value_pair_delimiter</span><span class="p">,</span> <span class="n">absl</span><span class="o">::</span><span class="n">SkipEmpty</span><span class="p">()))</span> <span class="p">{</span>
      <span class="k">typename</span> <span class="n">SplitString2ValueVecCallbackArgsType</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;::</span><span class="n">type</span> <span class="n">tuple</span><span class="p">;</span>
      <span class="k">using</span> <span class="n">element_type</span> <span class="o">=</span> <span class="k">typename</span> <span class="n">std</span><span class="o">::</span><span class="n">tuple_element</span><span class="o">&lt;</span><span class="mi">0</span><span class="p">,</span> <span class="k">decltype</span><span class="p">(</span><span class="n">tuple</span><span class="p">)</span><span class="o">&gt;::</span><span class="n">type</span><span class="p">;</span>
      <span class="kt">bool</span> <span class="n">ok</span>            <span class="o">=</span> <span class="nb">true</span><span class="p">;</span>
      <span class="k">for</span> <span class="p">(</span><span class="k">const</span> <span class="k">auto</span> <span class="o">&amp;</span><span class="n">item</span> <span class="o">:</span> <span class="n">absl</span><span class="o">::</span><span class="n">StrSplit</span><span class="p">(</span><span class="n">kv_token</span><span class="p">,</span> <span class="n">key_value_delimiter</span><span class="p">))</span> <span class="p">{</span>
        <span class="k">typename</span> <span class="n">element_type</span><span class="o">::</span><span class="n">value_type</span> <span class="n">v</span><span class="p">;</span>
        <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">Conv2Type</span><span class="o">&lt;</span><span class="k">decltype</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">&gt;</span> <span class="p">{}(</span><span class="n">item</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">v</span><span class="p">))</span> <span class="p">{</span>
          <span class="n">ok</span> <span class="o">=</span> <span class="nb">false</span><span class="p">;</span>
          <span class="k">break</span><span class="p">;</span>
        <span class="p">}</span>
        <span class="n">std</span><span class="o">::</span><span class="n">get</span><span class="o">&lt;</span><span class="mi">0</span><span class="o">&gt;</span><span class="p">(</span><span class="n">tuple</span><span class="p">).</span><span class="n">push_back</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">v</span><span class="p">));</span>
      <span class="p">}</span>
      <span class="k">if</span> <span class="p">(</span><span class="n">ok</span><span class="p">)</span> <span class="p">{</span>
        <span class="n">ApplyInvoke</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;</span> <span class="p">{}(</span><span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;</span><span class="p">(</span><span class="n">cb</span><span class="p">),</span> <span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">tuple</span><span class="p">));</span>
      <span class="p">}</span>
    <span class="p">}</span>
  <span class="p">}</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">Callback</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">STRING</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">SplitString2ValueVec</span><span class="o">&lt;</span>
    <span class="n">Callback</span><span class="p">,</span> <span class="n">STRING</span><span class="p">,</span>
    <span class="k">typename</span> <span class="n">std</span><span class="o">::</span><span class="n">enable_if</span><span class="o">&lt;</span>
        <span class="n">IsCallbackArgsOnly1Vec</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;::</span><span class="n">value</span> <span class="o">&amp;&amp;</span> <span class="c1">// 参数为std::vector&lt;T&gt;时</span>
        <span class="n">std</span><span class="o">::</span><span class="n">is_same</span><span class="o">&lt;</span><span class="kt">bool</span><span class="p">,</span> <span class="n">boost</span><span class="o">::</span><span class="n">callable_traits</span><span class="o">::</span><span class="n">return_type_t</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;&gt;::</span><span class="n">value</span><span class="o">&gt;::</span><span class="n">type</span><span class="o">&gt;</span> <span class="p">{</span>
  <span class="c1">// 返回值为bool类型的回调函数时，返回值为false时，中断解析。</span>
  <span class="kt">void</span> <span class="k">operator</span><span class="p">()(</span><span class="k">const</span> <span class="n">STRING</span> <span class="o">&amp;</span><span class="n">v</span><span class="p">,</span> <span class="kt">char</span> <span class="n">key_value_pair_delimiter</span><span class="p">,</span> <span class="kt">char</span> <span class="n">key_value_delimiter</span><span class="p">,</span>
                  <span class="n">Callback</span> <span class="o">&amp;&amp;</span><span class="n">cb</span><span class="p">)</span> <span class="p">{</span>
    <span class="k">for</span> <span class="p">(</span><span class="k">const</span> <span class="k">auto</span> <span class="o">&amp;</span><span class="n">kv_token</span> <span class="o">:</span> <span class="n">absl</span><span class="o">::</span><span class="n">StrSplit</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">key_value_pair_delimiter</span><span class="p">,</span> <span class="n">absl</span><span class="o">::</span><span class="n">SkipEmpty</span><span class="p">()))</span> <span class="p">{</span>
      <span class="k">typename</span> <span class="n">SplitString2ValueVecCallbackArgsType</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;::</span><span class="n">type</span> <span class="n">tuple</span><span class="p">;</span>
      <span class="k">using</span> <span class="n">element_type</span> <span class="o">=</span> <span class="k">typename</span> <span class="n">std</span><span class="o">::</span><span class="n">tuple_element</span><span class="o">&lt;</span><span class="mi">0</span><span class="p">,</span> <span class="k">decltype</span><span class="p">(</span><span class="n">tuple</span><span class="p">)</span><span class="o">&gt;::</span><span class="n">type</span><span class="p">;</span>
      <span class="kt">bool</span> <span class="n">ok</span>            <span class="o">=</span> <span class="nb">true</span><span class="p">;</span>
      <span class="k">for</span> <span class="p">(</span><span class="k">const</span> <span class="k">auto</span> <span class="o">&amp;</span><span class="n">item</span> <span class="o">:</span> <span class="n">absl</span><span class="o">::</span><span class="n">StrSplit</span><span class="p">(</span><span class="n">kv_token</span><span class="p">,</span> <span class="n">key_value_delimiter</span><span class="p">))</span> <span class="p">{</span>
        <span class="k">typename</span> <span class="n">element_type</span><span class="o">::</span><span class="n">value_type</span> <span class="n">v</span><span class="p">;</span>
        <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">Conv2Type</span><span class="o">&lt;</span><span class="k">decltype</span><span class="p">(</span><span class="n">v</span><span class="p">)</span><span class="o">&gt;</span> <span class="p">{}(</span><span class="n">item</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">v</span><span class="p">))</span> <span class="p">{</span>
          <span class="n">ok</span> <span class="o">=</span> <span class="nb">false</span><span class="p">;</span>
          <span class="k">break</span><span class="p">;</span>
        <span class="p">}</span>
        <span class="n">std</span><span class="o">::</span><span class="n">get</span><span class="o">&lt;</span><span class="mi">0</span><span class="o">&gt;</span><span class="p">(</span><span class="n">tuple</span><span class="p">).</span><span class="n">push_back</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">v</span><span class="p">));</span>
      <span class="p">}</span>
      <span class="k">if</span> <span class="p">(</span><span class="n">ok</span><span class="p">)</span> <span class="p">{</span>
        <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">ApplyInvoke</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;</span> <span class="p">{}(</span><span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;</span><span class="p">(</span><span class="n">cb</span><span class="p">),</span> <span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">tuple</span><span class="p">)))</span> <span class="p">{</span>
          <span class="k">break</span><span class="p">;</span>
        <span class="p">}</span>
      <span class="p">}</span>
    <span class="p">}</span>
  <span class="p">}</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">Callback</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">StringType</span><span class="p">&gt;</span>
<span class="kr">inline</span> <span class="kt">void</span> <span class="n">SplitString2ValueVecF</span><span class="p">(</span><span class="k">const</span> <span class="n">StringType</span> <span class="o">&amp;</span><span class="n">v</span><span class="p">,</span> <span class="kt">char</span> <span class="n">vec_vec_delimiter</span><span class="p">,</span> <span class="kt">char</span> <span class="n">value_delimiter</span><span class="p">,</span>
                                  <span class="n">Callback</span> <span class="o">&amp;&amp;</span><span class="n">cb</span><span class="p">)</span> <span class="p">{</span>
  <span class="n">SplitString2ValueVec</span><span class="o">&lt;</span><span class="n">Callback</span><span class="p">,</span> <span class="n">StringType</span><span class="p">,</span>
                       <span class="k">typename</span> <span class="n">SplitString2ValueVecCallbackReturnType</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;::</span><span class="n">type</span><span class="o">&gt;</span> <span class="p">{}(</span>
      <span class="n">v</span><span class="p">,</span> <span class="n">vec_vec_delimiter</span><span class="p">,</span> <span class="n">value_delimiter</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;</span><span class="p">(</span><span class="n">cb</span><span class="p">));</span>
<span class="p">}</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">Callback</span><span class="p">&gt;</span>
<span class="kr">inline</span> <span class="kt">void</span> <span class="n">SplitString2ValueVecF</span><span class="p">(</span><span class="k">const</span> <span class="n">butil</span><span class="o">::</span><span class="n">StringPiece</span> <span class="o">&amp;</span><span class="n">v</span><span class="p">,</span> <span class="kt">char</span> <span class="n">vec_vec_delimiter</span><span class="p">,</span>
                                  <span class="kt">char</span> <span class="n">value_delimiter</span><span class="p">,</span> <span class="n">Callback</span> <span class="o">&amp;&amp;</span><span class="n">cb</span><span class="p">)</span> <span class="p">{</span>
  <span class="n">SplitString2ValueVecF</span><span class="o">&lt;</span><span class="n">Callback</span><span class="p">,</span> <span class="n">absl</span><span class="o">::</span><span class="n">string_view</span><span class="o">&gt;</span><span class="p">(</span><span class="n">absl</span><span class="o">::</span><span class="n">string_view</span> <span class="p">{</span><span class="n">v</span><span class="p">.</span><span class="n">data</span><span class="p">(),</span> <span class="n">v</span><span class="p">.</span><span class="n">length</span><span class="p">()},</span>
                                                     <span class="n">vec_vec_delimiter</span><span class="p">,</span> <span class="n">value_delimiter</span><span class="p">,</span>
                                                     <span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;</span><span class="p">(</span><span class="n">cb</span><span class="p">));</span>
<span class="p">}</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">Callback</span><span class="p">&gt;</span>
<span class="kr">inline</span> <span class="kt">void</span> <span class="n">SplitString2ValueVecF</span><span class="p">(</span><span class="k">const</span> <span class="n">boost</span><span class="o">::</span><span class="n">string_view</span> <span class="o">&amp;</span><span class="n">v</span><span class="p">,</span> <span class="kt">char</span> <span class="n">vec_vec_delimiter</span><span class="p">,</span>
                                  <span class="kt">char</span> <span class="n">value_delimiter</span><span class="p">,</span> <span class="n">Callback</span> <span class="o">&amp;&amp;</span><span class="n">cb</span><span class="p">)</span> <span class="p">{</span>
  <span class="n">SplitString2ValueVecF</span><span class="o">&lt;</span><span class="n">Callback</span><span class="p">,</span> <span class="n">absl</span><span class="o">::</span><span class="n">string_view</span><span class="o">&gt;</span><span class="p">(</span><span class="n">absl</span><span class="o">::</span><span class="n">string_view</span> <span class="p">{</span><span class="n">v</span><span class="p">.</span><span class="n">data</span><span class="p">(),</span> <span class="n">v</span><span class="p">.</span><span class="n">length</span><span class="p">()},</span>
                                                     <span class="n">vec_vec_delimiter</span><span class="p">,</span> <span class="n">value_delimiter</span><span class="p">,</span>
                                                     <span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;</span><span class="p">(</span><span class="n">cb</span><span class="p">));</span>
<span class="p">}</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">MAP</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">STRING</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">SplitString2KeyValueMap</span> <span class="p">{</span>
  <span class="kt">void</span> <span class="k">operator</span><span class="p">()(</span><span class="k">const</span> <span class="n">STRING</span> <span class="o">&amp;</span><span class="n">v</span><span class="p">,</span> <span class="kt">char</span> <span class="n">key_value_pair_delimiter</span><span class="p">,</span> <span class="kt">char</span> <span class="n">key_value_delimiter</span><span class="p">,</span>
                  <span class="n">MAP</span> <span class="o">&amp;</span><span class="n">map</span><span class="p">)</span> <span class="p">{</span>
    <span class="k">static_assert</span><span class="p">(</span><span class="n">common_detail</span><span class="o">::</span><span class="n">has_key_type_traits</span><span class="o">&lt;</span><span class="n">MAP</span><span class="o">&gt;::</span><span class="n">value</span><span class="p">,</span>
                  <span class="s">"map type must have key_type alias."</span><span class="p">);</span>
    <span class="k">static_assert</span><span class="p">(</span><span class="n">absl</span><span class="o">::</span><span class="n">strings_internal</span><span class="o">::</span><span class="n">HasMappedType</span><span class="o">&lt;</span><span class="n">MAP</span><span class="o">&gt;::</span><span class="n">value</span><span class="p">,</span>
                  <span class="s">"map type must have mapped_type alias."</span><span class="p">);</span>
    <span class="k">for</span> <span class="p">(</span><span class="k">const</span> <span class="k">auto</span> <span class="o">&amp;</span><span class="n">kv_token</span> <span class="o">:</span> <span class="n">absl</span><span class="o">::</span><span class="n">StrSplit</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">key_value_pair_delimiter</span><span class="p">,</span> <span class="n">absl</span><span class="o">::</span><span class="n">SkipEmpty</span><span class="p">()))</span> <span class="p">{</span>
      <span class="n">std</span><span class="o">::</span><span class="n">pair</span><span class="o">&lt;</span><span class="n">absl</span><span class="o">::</span><span class="n">string_view</span><span class="p">,</span> <span class="n">absl</span><span class="o">::</span><span class="n">string_view</span><span class="o">&gt;</span> <span class="n">kv</span> <span class="o">=</span>
          <span class="n">absl</span><span class="o">::</span><span class="n">StrSplit</span><span class="p">(</span><span class="n">kv_token</span><span class="p">,</span> <span class="n">absl</span><span class="o">::</span><span class="n">MaxSplits</span><span class="p">(</span><span class="n">key_value_delimiter</span><span class="p">,</span> <span class="mi">1</span><span class="p">));</span>
      <span class="k">typename</span> <span class="n">MAP</span><span class="o">::</span><span class="n">key_type</span>    <span class="n">key</span> <span class="o">=</span> <span class="p">{};</span>
      <span class="k">typename</span> <span class="n">MAP</span><span class="o">::</span><span class="n">mapped_type</span> <span class="n">val</span> <span class="o">=</span> <span class="p">{};</span>
      <span class="k">if</span> <span class="p">(</span><span class="n">common_detail</span><span class="o">::</span><span class="n">Conv2Type</span><span class="o">&lt;</span><span class="k">decltype</span><span class="p">(</span><span class="n">key</span><span class="p">)</span><span class="o">&gt;</span><span class="p">()(</span><span class="n">kv</span><span class="p">.</span><span class="n">first</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">key</span><span class="p">)</span> <span class="o">&amp;&amp;</span>
          <span class="n">common_detail</span><span class="o">::</span><span class="n">Conv2Type</span><span class="o">&lt;</span><span class="k">decltype</span><span class="p">(</span><span class="n">val</span><span class="p">)</span><span class="o">&gt;</span><span class="p">()(</span><span class="n">kv</span><span class="p">.</span><span class="n">second</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">val</span><span class="p">))</span> <span class="p">{</span>
        <span class="n">map</span><span class="p">.</span><span class="n">emplace</span><span class="p">(</span><span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">key</span><span class="p">),</span> <span class="n">std</span><span class="o">::</span><span class="n">move</span><span class="p">(</span><span class="n">val</span><span class="p">));</span>
      <span class="p">}</span>
    <span class="p">}</span>
  <span class="p">}</span>
<span class="p">};</span>

<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">MAP</span><span class="p">&gt;</span>
<span class="k">struct</span> <span class="nc">SplitString2KeyValueMap</span><span class="o">&lt;</span><span class="n">MAP</span><span class="p">,</span> <span class="n">butil</span><span class="o">::</span><span class="n">StringPiece</span><span class="o">&gt;</span> <span class="p">{</span>
  <span class="kt">void</span> <span class="k">operator</span><span class="p">()(</span><span class="k">const</span> <span class="n">butil</span><span class="o">::</span><span class="n">StringPiece</span> <span class="o">&amp;</span><span class="n">v</span><span class="p">,</span> <span class="kt">char</span> <span class="n">key_value_pair_delimiter</span><span class="p">,</span>
                  <span class="kt">char</span> <span class="n">key_value_delimiter</span><span class="p">,</span> <span class="n">MAP</span> <span class="o">&amp;</span><span class="n">map</span><span class="p">)</span> <span class="p">{</span>
    <span class="n">SplitString2KeyValueMap</span><span class="o">&lt;</span><span class="n">MAP</span><span class="p">,</span> <span class="n">absl</span><span class="o">::</span><span class="n">string_view</span><span class="o">&gt;</span><span class="p">()(</span>
        <span class="n">absl</span><span class="o">::</span><span class="n">string_view</span> <span class="p">{</span><span class="n">v</span><span class="p">.</span><span class="n">data</span><span class="p">(),</span> <span class="n">v</span><span class="p">.</span><span class="n">size</span><span class="p">()},</span> <span class="n">key_value_pair_delimiter</span><span class="p">,</span> <span class="n">key_value_delimiter</span><span class="p">,</span> <span class="n">map</span><span class="p">);</span>
  <span class="p">}</span>
<span class="p">};</span>
<span class="p">}</span> <span class="c1">// namespace common_detail</span>

<span class="c1">// SplitString2ValueVec 可以将字符串按照回调lambda表达的的输入参数的类型，按需分段解析成对应的</span>
<span class="c1">// 类型，然后调用对应的lambda表达式。可以极大地简化字符串解析。</span>
<span class="c1">//</span>
<span class="c1">// vec_vec_delimiter 为外层字符串格式的分隔符</span>
<span class="c1">// value_delimiter 为内层字符串格式的分隔符</span>
<span class="c1">//</span>
<span class="c1">// 例如：当 vec_vec_delimiter='@',</span>
<span class="c1">// value_delimiter=','，被解析字符串为"Tom,1,0.5@Jame,2,0.8@Bean,3,1.0"， 回调函数为 :</span>
<span class="c1">// [](std::string name, int index, double score) {</span>
<span class="c1">//    std::cout &lt;&lt; "name: " &lt;&lt; name &lt;&lt; " index:" &lt;&lt; index &lt;&lt; " score:" &lt;&lt; score &lt;&lt; std::endl;</span>
<span class="c1">// }</span>
<span class="c1">// 时，则会将字符串解析为按外层分隔符'@'解析为3段，然后再按内层分隔符','将每一段解析成3个三个段，</span>
<span class="c1">// 然后分别转换为对应类型std::string int double，然后调用对应的回调函数。然后回调函数分别输出:</span>
<span class="c1">// name:Tom index:1 score:0.5</span>
<span class="c1">// name:Jame index:2 score:0.8</span>
<span class="c1">// name:Bean index:3 score:1.0</span>
<span class="c1">//</span>
<span class="c1">// 当第二层字符串为同一类型T且个数不定时，传入的lambda可以将接受的参数指定为对应类型的std::vector&lt;T&gt;，</span>
<span class="c1">// 这样该函数会将第二层字符串解析为对应类型并传入lambda。</span>
<span class="c1">//</span>
<span class="c1">// 例如:</span>
<span class="c1">// std::string xxx = "1.0@1.1@1.2|2.1@2.2@2.3@2.4";</span>
<span class="c1">// common::SplitString2ValueVec(xxx, '|', '@', [](std::vector&lt;double&gt; vec) {</span>
<span class="c1">//   std::cout &lt;&lt; "vec size=" &lt;&lt; vec.size();</span>
<span class="c1">//   for (const auto &amp; v : vec) {</span>
<span class="c1">//      std::cout &lt;&lt; " v=" &lt;&lt; v &lt;&lt; ",";</span>
<span class="c1">//   };</span>
<span class="c1">//   std::cout &lt;&lt; std::endl;</span>
<span class="c1">// });</span>
<span class="c1">// 则会输出：</span>
<span class="c1">// vec size=3 v=1.0 v=1.1 v=1.2</span>
<span class="c1">// vec size=4 v=2.1 v=2.2 v=2.3 v=2.4</span>
<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">Callback</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">StringType</span><span class="p">&gt;</span>
<span class="kr">inline</span> <span class="kt">void</span> <span class="n">SplitString2ValueVec</span><span class="p">(</span><span class="k">const</span> <span class="n">StringType</span> <span class="o">&amp;</span><span class="n">v</span><span class="p">,</span> <span class="kt">char</span> <span class="n">vec_vec_delimiter</span><span class="p">,</span> <span class="kt">char</span> <span class="n">value_delimiter</span><span class="p">,</span>
                                 <span class="n">Callback</span> <span class="o">&amp;&amp;</span><span class="n">cb</span><span class="p">)</span> <span class="p">{</span>
  <span class="n">common_detail</span><span class="o">::</span><span class="n">SplitString2ValueVecF</span><span class="p">(</span><span class="n">v</span><span class="p">,</span> <span class="n">vec_vec_delimiter</span><span class="p">,</span> <span class="n">value_delimiter</span><span class="p">,</span>
                                       <span class="n">std</span><span class="o">::</span><span class="n">forward</span><span class="o">&lt;</span><span class="n">Callback</span><span class="o">&gt;</span><span class="p">(</span><span class="n">cb</span><span class="p">));</span>
<span class="p">}</span>
<span class="c1">// 例如：当 key_value_pair_delimiter='@', key_value_delimiter='='，map=std::map&lt;std::string,</span>
<span class="c1">// int&gt;时， 将 "abc=1@efg=2@xyz=3" 拆解成 map = { {"abc", 1}, {"efg", 2}, {xyz, 3} }</span>
<span class="c1">// 支持任意map类型，比如:</span>
<span class="c1">//      std::map&lt;std::string, std::string&gt;</span>
<span class="c1">//      std::map&lt;int, int&gt;</span>
<span class="c1">//      std::unordered_map&lt;int, int&gt;</span>
<span class="c1">// 等</span>
<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">MAP</span><span class="p">,</span> <span class="k">typename</span> <span class="nc">STRING</span><span class="p">&gt;</span>
<span class="kr">inline</span> <span class="kt">void</span> <span class="n">SplitString2KeyValueMap</span><span class="p">(</span><span class="k">const</span> <span class="n">STRING</span> <span class="o">&amp;</span><span class="n">v</span><span class="p">,</span> <span class="kt">char</span> <span class="n">key_value_pair_delimiter</span><span class="p">,</span>
                                    <span class="kt">char</span> <span class="n">key_value_delimiter</span><span class="p">,</span> <span class="n">MAP</span> <span class="o">&amp;</span><span class="n">map</span><span class="p">)</span> <span class="p">{</span>
  <span class="n">common_detail</span><span class="o">::</span><span class="n">SplitString2KeyValueMap</span><span class="o">&lt;</span><span class="n">MAP</span><span class="p">,</span> <span class="n">STRING</span><span class="o">&gt;</span><span class="p">()(</span><span class="n">v</span><span class="p">,</span> <span class="n">key_value_pair_delimiter</span><span class="p">,</span>
                                                        <span class="n">key_value_delimiter</span><span class="p">,</span> <span class="n">map</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span> <span class="c1">// namespace common</span>

</code></pre></div></div>]]></content><author><name>Neal Hu</name></author><category term="c++" /><summary type="html"><![CDATA[c++]]></summary></entry><entry><title type="html">folly utility 简明摘要</title><link href="https://lrita.github.io/2021/06/11/cpp-folly-utility/" rel="alternate" type="text/html" title="folly utility 简明摘要" /><published>2021-06-11T00:00:00+08:00</published><updated>2021-06-11T00:00:00+08:00</updated><id>https://lrita.github.io/2021/06/11/cpp-folly-utility</id><content type="html" xml:base="https://lrita.github.io/2021/06/11/cpp-folly-utility/"><![CDATA[<h1 id="辅助类">辅助类</h1>

<h2 id="functional">functional</h2>

<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// 所在头文件：</span>
<span class="cp">#include</span> <span class="cpf">&lt;folly/functional/Partial.h&gt;</span><span class="cp">
</span>
<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">F</span><span class="p">,</span> <span class="k">typename</span><span class="o">...</span> <span class="nc">Args</span><span class="p">&gt;</span>
<span class="k">auto</span> <span class="nf">partial</span><span class="p">(</span><span class="n">F</span><span class="o">&amp;&amp;</span> <span class="n">f</span><span class="p">,</span> <span class="n">Args</span><span class="o">&amp;&amp;</span><span class="p">...</span> <span class="n">args</span><span class="p">);</span>

<span class="c1">// 跟 std::bind() 相似，但是不需要使用placeholders来进行参数绑定。</span>
<span class="c1">// e.g.</span>
<span class="c1">//</span>
<span class="c1">// auto p = folly::partial(&amp;Foo::method, foo_pointer);</span>
<span class="c1">// p();</span>
<span class="c1">//</span>
<span class="c1">// folly::partial(Foo, 1, 2)(3); // is equivalent to `Foo(1, 2, 3);`</span>
</code></pre></div></div>

<h2 id="编译期数学方法">编译期数学方法</h2>

<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// 所在头文件：</span>
<span class="cp">#include</span> <span class="cpf">&lt;folly/ConstexprMath.h&gt;</span><span class="cp">
</span>
<span class="c1">// 其中包含一些模板类，可以帮助在编译期推导 max、min、abs、pow、ceil、log2_ceil、log2等常用数学方法。</span>
</code></pre></div></div>

<h1 id="线程同步">线程同步</h1>

<h2 id="spin-帮助方法">spin 帮助方法</h2>

<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">#include</span> <span class="cpf">&lt;folly/synchronization/detail/Spin.h&gt;</span><span class="cp">
</span>
<span class="c1">// 非常简单易用的 spin 循环实现，需要用到的之后可以直接引用</span>
<span class="n">folly</span><span class="o">::</span><span class="n">detail</span><span class="o">::</span><span class="n">spin_pause_until</span><span class="p">();</span>
<span class="n">folly</span><span class="o">::</span><span class="n">detail</span><span class="o">::</span><span class="n">spin_yield_until</span><span class="p">();</span>
</code></pre></div></div>

<h2 id="follymicrolock">folly::MicroLock</h2>

<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">#include</span> <span class="cpf">&lt;folly/MicroLock.h&gt;</span><span class="cp">
</span><span class="c1">// 或者</span>
<span class="cp">#include</span> <span class="cpf">&lt;folly/synchronization/SmallLocks.h&gt;</span><span class="cp">
</span>
<span class="c1">// 通常从性能角度出发，应该使用 std::mutex（对锁竞争处理的更好）， 如果为了节省内存，可以使用 folly::MicroLock ，</span>
<span class="c1">// 其只使用 4B 空间。</span>
<span class="c1">// 其具有常规的lock()/try_lock()/unlock()方法。</span>
</code></pre></div></div>

<h2 id="follymicrospinlockfollypicospinlockfollyrwspinlock">folly::MicroSpinLock、folly::PicoSpinLock、folly::RWSpinLock</h2>
<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">#include</span> <span class="cpf">&lt;folly/MicroSpinLock.h&gt;</span><span class="cp">
</span><span class="c1">// 或者</span>
<span class="cp">#include</span> <span class="cpf">&lt;folly/synchronization/MicroSpinLock.h&gt;</span><span class="cp">
</span>
<span class="c1">// folly::MicroSpinLock 是一个极简单的 spinlock 实现，采用最简单的while-cas模式。通常不应该被使用。</span>

<span class="cp">#include</span> <span class="cpf">&lt;folly/synchronization/PicoSpinLock.h&gt;</span><span class="cp">
</span>
<span class="c1">// folly::PicoSpinLock 也是一个最简单的while-cas模式实现的spinlock，通常不应该被使用。但是其使用一个整形数</span>
<span class="c1">// 作为spin的载体，其大部分bit可以用来存储数据，剩余bit用来记录lock状态，如果非常需要内存紧凑，可以考虑使用。</span>

<span class="cp">#include</span> <span class="cpf">&lt;folly/synchronization/RWSpinLock.h&gt;</span><span class="cp">
</span>
<span class="c1">// folly::RWSpinLock 是一个简单的读写spinlock实现，并且支持锁升级、降级逻辑，其升降级遵循boost的 https://www.boost.org/doc/libs/1_47_0/doc/html/thread/synchronization.html#thread.synchronization.mutex_concepts.upgrade_lockable 语义，其可以通过 lock_upgrade() + unlock_upgrade_and_lock() 完成读锁到写锁的升级。</span>
</code></pre></div></div>

<p>需要明确的是，spinlock类的锁都不太适合在应用层面随意使用，你必须明显进行测试、并且清楚自己了解内在机理，否则你只应该简单的使用 <code class="language-plaintext highlighter-rouge">std::mutex</code>或者<code class="language-plaintext highlighter-rouge">folly::SharedMutex</code>等，可以参考<a href="https://matklad.github.io//2020/01/02/spinlocks-considered-harmful.html">Spinlocks Considered Harmful</a>等。</p>

<h2 id="baton">baton</h2>

<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// 所在头文件：</span>
<span class="cp">#include</span> <span class="cpf">&lt;folly/synchronization/Baton.h&gt;</span><span class="cp">
</span>
<span class="c1">// Baton 通常用作线程间同步、等待、通知的标识符号，常用姿势是，一些线程调用 wait() 方法等待另</span>
<span class="c1">// 一些线程完成某项工作，其完成以后调用 post() 方法进行通知。 其跟一般PV信号量的区别是，Baton</span>
<span class="c1">// 更轻量化、通知策略更简单(没有FILO/FIFO等策略)、仅能够通知一次，在简单场景中更高效。</span>
<span class="c1">//</span>
<span class="c1">// 声明：MayBlock表示十分会被长时间block，其实就是内部衡量是否需要一直spin的依据，</span>
<span class="c1">// 否则会调用 futex 相关 syscall 释放 cpu</span>
<span class="c1">// Atom 指示使用什么原子操作的实现，通常使用 std::atomic 即可</span>
<span class="k">template</span> <span class="o">&lt;</span><span class="kt">bool</span> <span class="n">MayBlock</span> <span class="o">=</span> <span class="nb">true</span><span class="p">,</span> <span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span><span class="p">&gt;</span> <span class="k">class</span> <span class="nc">Atom</span> <span class="o">=</span> <span class="n">std</span><span class="o">::</span><span class="n">atomic</span><span class="p">&gt;</span>
<span class="k">class</span> <span class="nc">Baton</span><span class="p">;</span>

<span class="c1">// 常规用法：</span>
<span class="c1">// Baton 用于同步线程间的PV (block/wakeup)，但是不像 semaphores</span>
<span class="c1">// 信号量那样可以多次pv，baton 仅支持单次pv操作，folly::Future 中的</span>
<span class="c1">// block/wakeup 就是使用 Baton&lt;&gt; 来实现的。</span>

<span class="n">Baton</span><span class="o">&lt;&gt;</span> <span class="n">baton</span><span class="p">;</span>
<span class="n">Baton</span><span class="o">&lt;</span><span class="nb">false</span><span class="o">&gt;</span> <span class="n">spin_baton</span><span class="p">;</span>

<span class="c1">// 其基本方法有：</span>
<span class="kt">bool</span> <span class="n">Baton</span><span class="o">&lt;&gt;::</span><span class="n">ready</span><span class="p">();</span> <span class="c1">// 测试是否已经被标记置位</span>
<span class="kt">void</span> <span class="n">Baton</span><span class="o">&lt;&gt;::</span><span class="n">post</span><span class="p">();</span>  <span class="c1">// 置位</span>
<span class="kt">bool</span> <span class="n">Baton</span><span class="o">&lt;&gt;::</span><span class="n">try_wait</span><span class="p">();</span> <span class="c1">// 等同 ready</span>
<span class="c1">// 等待直到被置位，可以传入一个 wait_options 来控制 spin 的最大时间，默认2us</span>
<span class="kt">void</span> <span class="n">Baton</span><span class="o">&lt;&gt;::</span><span class="n">wait</span><span class="p">(</span><span class="k">const</span> <span class="n">WaitOptions</span><span class="o">&amp;</span> <span class="n">opt</span> <span class="o">=</span> <span class="n">wait_options</span><span class="p">());</span>
</code></pre></div></div>

<h2 id="follysaturatingsemaphore">folly::SaturatingSemaphore</h2>

<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// 所在头文件：</span>
<span class="cp">#include</span> <span class="cpf">&lt;folly/synchronization/SaturatingSemaphore.h&gt;</span><span class="cp">
</span>
<span class="c1">// folly::SaturatingSemaphore 是一个经典的信号量实现，可以支持多个poster和多个waiter同时调用PV。</span>
<span class="c1">// 而且可以对一个SaturatingSemaphore可以多次设置post状态，而且幂等（但是不累积，设置成post状态后，</span>
<span class="c1">// 所有wait()调用都会通过，简而言之就是只有PV状态，而没有对应的计数），然后可以通过reset()方法进行恢复。</span>
<span class="c1">//</span>
<span class="c1">// 其跟Baton的主要区别就是Baton只支持有且仅有一个poster，SaturatingSemaphore支持多个，且可以并发调用post()</span>

<span class="c1">///  方法：</span>
<span class="c1">///   bool ready():</span>
<span class="c1">///     Returns true if the flag is set by a call to post, otherwise false.</span>
<span class="c1">///     Equivalent to try_wait, but available on const receivers.</span>
<span class="c1">///   void reset();</span>
<span class="c1">///     Clears the flag.</span>
<span class="c1">///   void post();</span>
<span class="c1">///     Sets the flag and wakes all current waiters, i.e., causes all</span>
<span class="c1">///     concurrent calls to wait, try_wait_for, and try_wait_until to</span>
<span class="c1">///     return.</span>
<span class="c1">///   void wait(</span>
<span class="c1">///       WaitOptions opt = wait_options());</span>
<span class="c1">///     Waits for the flag to be set by a call to post.</span>
<span class="c1">///   bool try_wait();</span>
<span class="c1">///     Returns true if the flag is set by a call to post, otherwise false.</span>
<span class="c1">///   bool try_wait_until(</span>
<span class="c1">///       time_point&amp; deadline,</span>
<span class="c1">///       WaitOptions&amp; = wait_options());</span>
<span class="c1">///     Returns true if the flag is set by a call to post before the</span>
<span class="c1">///     deadline, otherwise false.</span>
<span class="c1">///   bool try_wait_for(</span>
<span class="c1">///       duration&amp;,</span>
<span class="c1">///       WaitOptions&amp; = wait_options());</span>
<span class="c1">///     Returns true if the flag is set by a call to post before the</span>
<span class="c1">///     expiration of the specified duration, otherwise false.</span>
</code></pre></div></div>

<h2 id="follylifosem">folly::LifoSem</h2>

<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">#include</span> <span class="cpf">&lt;folly/synchronization/LifoSem.h&gt;</span><span class="cp">
</span>
<span class="c1">// folly::LifoSem 相对于 folly::SaturatingSemaphore 维护了一个通知的顺序，后入先出，主要是尽快通知较活跃的线程。</span>
<span class="c1">// 并且post()可以指定通知的个数，不像 folly::SaturatingSemaphore post()会通知全部的wait().</span>
<span class="c1">//</span>
<span class="c1">// 其内部用一个对象池维护了waiter的相对顺序，然后按顺序进行通知。</span>
</code></pre></div></div>

<h2 id="hazard-pointer">Hazard Pointer</h2>

<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// 所在头文件:</span>
<span class="cp">#include</span> <span class="cpf">&lt;folly/synchronization/Hazptr.h&gt;</span><span class="cp">
</span>
<span class="c1">// hazard pointer 支持一写多读</span>
<span class="c1">// 其实现依赖thread local机制。</span>
<span class="c1">// 其原始的 hazard pointer 并没有直接暴露给用户来进行操作，而是给用户提供一个 hazptr_holder 对象进行持有 hazard pointer</span>
</code></pre></div></div>

<h1 id="atomic">Atomic</h1>

<h2 id="follyatomicstruct">folly::AtomicStruct</h2>

<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// AtomicStruct 可以原子地操作一个大小小于等于8byte的对象，其内部原理就是把对象转换成对应大小的int类型，</span>
<span class="c1">// 使用std::atmoic&lt;int&gt;来操作。是一个比较有助的封装。对象大小大于8byte的则无法使用该封装。</span>
<span class="cp">#include</span> <span class="cpf">&lt;folly/synchronization/AtomicStruct.h&gt;</span><span class="cp">
</span>
<span class="k">struct</span> <span class="nc">A</span> <span class="p">{</span>
  <span class="kt">int32_t</span> <span class="n">a</span><span class="p">;</span>
<span class="p">};</span>

<span class="n">folly</span><span class="o">::</span><span class="n">AtomicStruct</span><span class="o">&lt;</span><span class="n">A</span><span class="o">&gt;</span> <span class="n">a</span><span class="p">;</span>
<span class="k">auto</span> <span class="n">xx</span> <span class="o">=</span> <span class="n">a</span><span class="p">.</span><span class="n">load</span><span class="p">();</span>
</code></pre></div></div>

<h1 id="对象管理单例等">对象管理、单例等</h1>

<h2 id="线程维度的单例对象">线程维度的单例对象</h2>

<p>folly 的<code class="language-plaintext highlighter-rouge">folly/SingletonThreadLocal.h</code>文件中提供了宏<code class="language-plaintext highlighter-rouge">FOLLY_DECLARE_REUSED</code>以创建简便易用的线程维度的单例对象，可以减少逻辑上临时对象的反复创建、销毁的开销，要求该对象有一个<code class="language-plaintext highlighter-rouge">clear()</code>方法(通常是 STL 容器)即可。可以用这个宏优化线程池中对应函数中反复创建、销毁的临时容器。</p>

<p>其实现，就是创建一个对应对象的<code class="language-plaintext highlighter-rouge">thread_local</code>的单例，离开作用域的时候，调用对象的<code class="language-plaintext highlighter-rouge">clear()</code>方法：</p>

<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// in folly/SingletonThreadLocal.h</span>
<span class="cp">#define FOLLY_DECLARE_REUSED(name, ...)                                        \
  struct __folly_reused_type_##name {                                          \
    __VA_ARGS__ object;                                                        \
  };                                                                           \
  auto&amp; name =                                                                 \
      ::folly::SingletonThreadLocal&lt;__folly_reused_type_##name&gt;::get().object; \
  auto __folly_reused_g_##name = ::folly::makeGuard([&amp;] { name.clear(); })
</span></code></pre></div></div>

<p>用法</p>

<div class="language-cpp highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">#include</span> <span class="cpf">&lt;folly/SingletonThreadLocal.h&gt;</span><span class="cp">
</span>
<span class="kt">void</span> <span class="nf">traverse_perform</span><span class="p">(</span><span class="kt">int</span> <span class="n">root</span><span class="p">);</span>
<span class="k">template</span> <span class="o">&lt;</span><span class="k">typename</span> <span class="nc">F</span><span class="p">&gt;</span>
<span class="kt">void</span> <span class="nf">traverse_each_child_r</span><span class="p">(</span><span class="kt">int</span> <span class="n">root</span><span class="p">,</span> <span class="n">F</span> <span class="k">const</span><span class="o">&amp;</span><span class="p">);</span>
<span class="kt">void</span> <span class="nf">traverse_depthwise</span><span class="p">(</span><span class="kt">int</span> <span class="n">root</span><span class="p">)</span> <span class="p">{</span>
  <span class="c1">// preserves some of the memory backing these per-thread data structures</span>
  <span class="n">FOLLY_DECLARE_REUSED</span><span class="p">(</span><span class="n">seen</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">unordered_set</span><span class="o">&lt;</span><span class="kt">int</span><span class="o">&gt;</span><span class="p">);</span>
  <span class="n">FOLLY_DECLARE_REUSED</span><span class="p">(</span><span class="n">work</span><span class="p">,</span> <span class="n">std</span><span class="o">::</span><span class="n">vector</span><span class="o">&lt;</span><span class="kt">int</span><span class="o">&gt;</span><span class="p">);</span>
  <span class="c1">// example algorithm that uses these per-thread data structures</span>
  <span class="n">work</span><span class="p">.</span><span class="n">push_back</span><span class="p">(</span><span class="n">root</span><span class="p">);</span>
  <span class="k">while</span> <span class="p">(</span><span class="o">!</span><span class="n">work</span><span class="p">.</span><span class="n">empty</span><span class="p">())</span> <span class="p">{</span>
    <span class="n">root</span> <span class="o">=</span> <span class="n">work</span><span class="p">.</span><span class="n">back</span><span class="p">();</span>
    <span class="n">work</span><span class="p">.</span><span class="n">pop_back</span><span class="p">();</span>
    <span class="n">seen</span><span class="p">.</span><span class="n">insert</span><span class="p">(</span><span class="n">root</span><span class="p">);</span>
    <span class="n">traverse_perform</span><span class="p">(</span><span class="n">root</span><span class="p">);</span>
    <span class="n">traverse_each_child_r</span><span class="p">(</span><span class="n">root</span><span class="p">,</span> <span class="p">[</span><span class="o">&amp;</span><span class="p">](</span><span class="kt">int</span> <span class="n">item</span><span class="p">)</span> <span class="p">{</span>
      <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">seen</span><span class="p">.</span><span class="n">count</span><span class="p">(</span><span class="n">item</span><span class="p">))</span> <span class="p">{</span>
        <span class="n">work</span><span class="p">.</span><span class="n">push_back</span><span class="p">(</span><span class="n">item</span><span class="p">);</span>
      <span class="p">}</span>
    <span class="p">});</span>
  <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>]]></content><author><name>Neal Hu</name></author><category term="c++" /><summary type="html"><![CDATA[c++]]></summary></entry></feed>