New Two-Level L1 Data Cache Bypassing Technique for High Performance GPUs


Gwang Bok Kim, Cheol Hong Kim, Journal of Information Processing Systems Vol. 17, No. 1, pp. 51-62, Feb. 2021  

10.3745/JIPS.01.0062
Keywords: Bypassing, Cache, GPU, Miss Rate, Performance
Fulltext:

Abstract

On-chip caches of graphics processing units (GPUs) have contributed to improved GPU performance by reducing long memory access latency. However, cache efficiency remains low despite the facts that recent GPUs have considerably mitigated the bottleneck problem of L1 data cache. Although the cache miss rate is a reasonable metric for cache efficiency, it is not necessarily proportional to GPU performance. In this study, we introduce a second key determinant to overcome the problem of predicting the performance gains from L1 data cache based on the assumption that miss rate only is not accurate. The proposed technique estimates the benefits of the cache by measuring the balance between cache efficiency and throughput. The throughput of the cache is predicted based on the warp occupancy information in the warp pool. Then, the warp occupancy is used for a second bypass phase when workloads show an ambiguous miss rate. In our proposed architecture, the L1 data cache is turned off for a long period when the warp occupancy is not high. Our two-level bypassing technique can be applied to recent GPU models and improves the performance by 6% on average compared to the architecture without bypassing. Moreover, it outperforms the conventional bottleneck-based bypassing techniques.


Statistics
Show / Hide Statistics

Statistics (Cumulative Counts from November 1st, 2017)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.




Cite this article
[APA Style]
Gwang Bok Kim and Cheol Hong Kim (2021). New Two-Level L1 Data Cache Bypassing Technique for High Performance GPUs. Journal of Information Processing Systems, 17(1), 51-62. DOI: 10.3745/JIPS.01.0062.

[IEEE Style]
G. B. Kim and C. H. Kim, "New Two-Level L1 Data Cache Bypassing Technique for High Performance GPUs," Journal of Information Processing Systems, vol. 17, no. 1, pp. 51-62, 2021. DOI: 10.3745/JIPS.01.0062.

[ACM Style]
Gwang Bok Kim and Cheol Hong Kim. 2021. New Two-Level L1 Data Cache Bypassing Technique for High Performance GPUs. Journal of Information Processing Systems, 17, 1, (2021), 51-62. DOI: 10.3745/JIPS.01.0062.