使用内存映射文件来提高你程序的性能
2023-09-27 14:27:56 时间
本人在学习《WINDOWS核心编程》的时候对JEFFREY大师提到的一个小程序写了两个版本来比较性能,该程序的原始需求是这样的:对一个大文件进行倒序,也就是将一个文件头变成尾,尾变成头。
使用的方法有很多种,这里使用两个方法来比较,主要是突出使用内存映射文件好处;两种方法为:内存映射文件方法,I/O读写的缓存办法。第一种办法是创建内存映射文件对象,然后将该对象映射到进程的地址空间中,再读取文件内容,然后倒序,再写入文件。
第二中方法是,将文件内容读入一个大的缓冲区,然后倒序,再写入文件,中间对原来的文件删除,然后重新写入。
程序编写如下
第一种方法,内存映射文件方式:
![None.gif](http://www.cppblog.com/Images/OutliningIndicators/None.gif)
![ExpandedBlockStart.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedBlockStart.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockStart.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockStart.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockEnd.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockEnd.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockStart.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockStart.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockEnd.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockEnd.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockStart.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockStart.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockEnd.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockEnd.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockStart.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockStart.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockEnd.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockEnd.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedBlockEnd.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedBlockEnd.gif)
![None.gif](http://www.cppblog.com/Images/OutliningIndicators/None.gif)
第二中方法,使用缓存的方式:
![None.gif](http://www.cppblog.com/Images/OutliningIndicators/None.gif)
![ExpandedBlockStart.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedBlockStart.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockStart.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockStart.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockEnd.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockEnd.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockStart.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockStart.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockStart.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockStart.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockEnd.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockEnd.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockStart.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockStart.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockEnd.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockEnd.gif)
![ExpandedSubBlockEnd.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockEnd.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockStart.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockStart.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedSubBlockEnd.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedSubBlockEnd.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![InBlock.gif](http://www.cppblog.com/Images/OutliningIndicators/InBlock.gif)
![ExpandedBlockEnd.gif](http://www.cppblog.com/Images/OutliningIndicators/ExpandedBlockEnd.gif)
我运行了几次,比较结果如下:
文件大小(byte) 1方法时间(ms) 2方法时间(ms)
本人测试机器的CPU是迅池1.5的笔记本,内存为712MB
通过上面的测试我们可以看到使用内存映射文件的好处,在文件内存越大这种优势就体现的越明显,其中主要的原因是:
内存映射文件直接将文件的地址映射到进程的地址空间中,那么操作文件就相当于在内存中操作一样,省去了读和写I/O的时间;第二种方式是必须这么做(READFILE,WRITEFILE),这个过程是很慢的。
存储使者,内存驿站-文件操作 磁盘上的文件是文件 但是在程序设计中,我们一般谈的文件有两种:程序文件,数据文件(从文件功能的角度来分类的)
(理论篇)从基础文件IO说起虚拟内存,内存文件映射,零拷贝 新IO采用了内存映射的方式来处理输入/输出,新IO将文件或文件的一段区域映射到内存中,这样就可以像访问内存一样访问文件了,通过这种方式比传统的输入/输出要快的多。通过内存映射机制操作文件比使用常规方法和使用FileChannel读写高效的多。
理解程序内存 内存对很多人来说感觉是个很熟悉的东西,因为我们在用VC调试程序时,很多时候都会察看内存中变量的值。但是,很多时候我们的思维也就因此局限在有源码的模块,当遇到一些跨模块或是没有源代码的Bug,我们还是无从下手。
相关文章
- 【VS开发】内存映射文件3
- 数据分析-第三方库(工具包):Numpy【使用ndarray对象处理多维数组】【比Python原生list运算效率高:①内存块风格;②支持并行化运算;③底层用C编写,内部解除了GIL(全局解释器锁)】
- 快速读取内存文件-内存映射文件的方法
- Java 可重入锁内存可见性分析
- 深入浅出 Java 中 JVM 内存管理
- php 如何在有限的内存中读取大文件
- 刘宏程:紫光华山全新Gen9引入永久性内存 存储性能提升34倍
- 读excel时候出现java内存溢出
- Linux下查看某一进程所占用内存的方法
- CPU高速缓存行与内存关系 及并发MESI 协议
- iOS开发篇-内存管理(上)
- Unity3D 的大场景内存优化
- 提升Android下内存的使用意识和排查能力
- Tensorflow2对GPU内存的分配策略
- 【Android 内存优化】使用 Memory Analyzer ( MAT ) 工具分析内存 ( hprof 文件转换 | MAT 工具下载 | MAT 工具使用 )
- 【转】C#大文件读取和查询--内存映射
- Nginx:移植Nginx内存池 | 附测试用例
- 内存问题排查手段及相关文件介绍
- c++windows内核编程笔记day12 硬盘逻辑分区管理、文件管理、内存管理