This commit re-pipelines ECC check logic in data cache and exception generate logic for better timing.
Now ecc error is checked 1 cycle after reading result from data sram. An extra cycle is added for load
writeback to ROB.

Future work: move the pipeline to https://github.com/OpenXiangShan/XiangShan/blob/master/src/main/scala/xiangshan/backend/CtrlBlock.scala#L266-L277, which add a regnext.

* dcache: repipeline ecc check logic for timing

* chore: fix normal loadAccessFault logic

* wbu: delay load unit wb for 1 cycle

* dcache: add 1 extra cycle for beu error report

* dcache: do not access plru when refill

Now we have accessed plru when load miss, we should not access plru
when refill

* dcache: not not access plru when miss queue full

It will help avoid invalid plru access when miss queue full

* feat: parameterize load/store pipeline, etc.

* fix: use LoadPipelineWidth rather than LoadQueueSize

* fix: parameterize `rdataPtrExtNext`

* SBuffer: fix idx update logic

* atomic: parameterize atomic logic in `MemBlock`

* StoreQueue: update allow enque requirement

* feat: support one load/store pipeline

* feat: parameterize `EnsbufferWidth`

* chore: resharp codes for better generated name

* chore: remove sc too many fail assertion

* chore: use XSWarn()

s1_tag_match_way is vaild iff tag_read.valid and meta_read.valid in s0
for the same req

* mem: optimize missq reject to lq timing

DCache replay request is quite slow to generate, as it need to compare
load address with address in all valid miss queue entries.

Now we delay the usage of replay request from data cache.
Now replay request will not influence normal execuation flow until
load_s3 (1 cycle after load_s2, load result writeback to RS).

It is worth mentioning that "select refilling inst for load
writeback" will be disabled if dcacheRequireReplay in the
last cycle.

* dcache: compare probe block addr instead of full addr

* mem: do not replay from RS when ldld vio or fwd failed

ld-ld violation or forward failure will let an normal load inst replay
from fetch. If TLB hit and ld-ld violation / forward failure happens,
we write back that inst immediately. Meanwhile, such insts will not be
replayed from rs.

It should fix "mem: optimize missq reject to lq timing"

* mem: fix replay from rs condition

* mem: reduce refill to use latency

This commit update lq entry flag carefully in load_s3 to avoid extra
refill delay. It will remove the extra refill delay introduced by #1375
without harming memblock timing.

In #1375, we delayed load refill when dcache miss queue entry fails
to accept a miss. #1375 exchanges performance for better timing.

* mem: fix rs feedback priority

When dataInvalid && mshrFull, a succeed refill should not cancel
rs replay.

dcache.resp.bits.miss used to depend on tag_error, it causes severe
timing problem. That dependence is now removed.

Now when tag_error, we:

* Set access fault bit in exception vec
* Do not update miss queue. That is to say, if miss, that inst
may not be refilled
* Mark that inst as dataForwarded so it will not wait for refill
* Report error to CSR and BEU

If tag_error come with a miss, writeback taht inst from load
queue. Otherwise, writeback it from load pipeline.

* mem: fix ldld vio mask gen logic

* mem: fix lq released flag update logic

Make sure that every load before a probe has correct released flag

See the PR of this commit for illustration

* mem: fix ld-ld violation check logic

* ci: clean up workspace before do real test

* mem: reduce lq released flag update delay for 1 cycle

* chore: bump difftest to run no-smp diff

* ci: add mc test

* mem: fix lq released flag update logic

* chore: set difftest firstCommit_limit to 10000

* ci: use dual-nemu-so for mc test

* dcache: fix bug that a block could be released twice

* MainPipe: fix bug in way_en of miss_req

* MainPipe: fix bug

ecc tag error should not be reported if we do not read tag

* mem: fix error csr update

* dcache: l2 error will now trigger atom error

* chore: fix cache error debug decoder

* mem: split L1CacheErrorInfo and L1BusErrorUnitInfo

* dcache: add source info in L1CacheErrorInfo

* ICache: fix valid signal and add source/opType

* dcache: fix bug in ecc error

* mem,csr: send full L1CacheErrorInfo to CSR

* icache: provide cache error info for CSR

* dcache: force resp hit if tag ecc error happens

* mem: reorg l1 cache error report path

Now dcache tag error will force trigger a hit

* dcache: fix readline ecc check error

* dcache: mainpipe will not be influenced by tag error

* dcache: fix data ecc check error

* dcache: if coh state is Nothing, do not raise error
Co-authored-by: Nzhanglinjuan <zhanglinjuan20s@ict.ac.cn>
Co-authored-by: NJinYue <jinyue20s@ict.ac.cn>

* mem: optimize missq reject to lq timing

DCache replay request is quite slow to generate, as it need to compare
load address with address in all valid miss queue entries.

Now we delay the usage of replay request from data cache.
Now replay request will not influence normal execution flow until
load_s3 (1 cycle after load_s2, load result writeback to RS).

Note1: It is worth mentioning that "select refilling inst for load
writeback" will be disabled if dcacheRequireReplay in the
last cycle.

Note2: ld-ld violation or forward failure will let an normal load inst replay
from fetch. If TLB hit and ld-ld violation / forward failure happens,
we write back that inst immediately. Meanwhile, such insts will not be
replayed from rs.

* dcache: compare probe block addr instead of full addr

* dcache: use sram to build ecc array

* MainPipe: latch s1_encTag to last until s1_fire
Authored-by: Nzhanglinjuan <zhanglinjuan20s@ict.ac.cn>

* dcache: let ecc error and l2 corrupt raise load af

If CSR.smblockctl.cache_error_enable is disabled, ecc error and l2 corrupt
will not raise any exception.

* mem: enable cache error by default

* mem: support store ecc check, add ecc error csr

Support store / atom ecc check (early version)
Add ecc error csr to distingush ecc error and other access fault

Timing opt and unit tests to be added.

* dcache: fix bug in ecc check

* dcache: remove redundant ecc array

* CacheInstruction: fix typo

* dcache: fix bugs in cache instruction on ecc

* MetaArray: wrap ecc array as a single module

* MissQueue: loose merging condition to ease timing stress

* MissQueue: remove grant_beats

* MissQueue: compare block addr, not the whole addr bits

* dcache: optimize timing for generating ready to sbuffer
Co-authored-by: NWilliam Wang <zeweiwang@outlook.com>

This commit optimizes the coding style and timing for hardware
performance counters.

By default, performance counters are RegNext(RegNext(_)).

Now we RegNext(refill_req) for 1 cycle. It will provide more
time for refillShouldBeBlocked calcuation

* dcache: give probe the highest priority

* dcache: fix block probe logic

* dcache: give replace_req higher priority

* mem: delay uncache op start for 1 cycle

* dcache: decouple miss and replay signal

Now resp.miss will not depend on s2_nack_no_mshr

* lq,mem: give released flag update 1 more cycle

* chore: fix a name typo

* dcache: delay probe req for 1 cycle

* Add 1 cycle in refill pipe
Co-authored-by: Nzhanglinjuan <zhanglinjuan20s@ict.ac.cn>

* sbuffer: do flush correctly while draining sbuffer

* ci: enable ci for timing-memblock branch

* mem: disable EnableFastForward for timing reasons

* sbuffer: optimize forward mask gen timing

* dcache: block main pipe req if refill req is valid

Refill req comes from refill arbiter. There is not time left for index
conflict check. Now we simplily block all main pipe req when refill
req comes from miss queue.

* dcache: delay some resp signals for better timing

* dcache: optimize wbq enq entry select timing

* WritebackQueue: optimize enqueue logic fir timing

* WritebackQueue: always reject a req when wbq is full

* Revert "ci: enable ci for timing-memblock branch"

This reverts commit 32453dc4.

* WritebackQueue: fix bug in secondary_valid
Co-authored-by: Nzhanglinjuan <zhanglinjuan20s@ict.ac.cn>

* dcache: merge replace pipe with main pipe for timing reason

* MainPipe: fix bug in s3_fire

* MainPipe: fix bug in delay_release sent to wbq

* MainPipe: fix bug in blocking policy

* MainPipe: send io.replace_resp in stage 3

* MainPipe: fix bug in miss_id sent to wbq

* MainPipe: fix bug
Co-authored-by: NWilliam Wang <zeweiwang@outlook.com>

* sbuffer: do flush correctly while draining sbuffer

* mem: disable EnableFastForward for timing reasons

* sbuffer: optimize forward mask gen timing

* dcache: block main pipe req if refill req is valid

Refill req comes from refill arbiter. There is not time left for index
conflict check. Now we block all main pipe req when refill
req comes from miss queue.

* dcache: delay some resp signals for better timing

* dcache: optimize wbq enq entry select timing

* dcache: decouple missq req.valid to valid & cancel

* valid is fast, it is used to select which miss req will be sent to
miss queue
* cancel can be slow to generate, it will cancel miss queue req in the
last moment

* sbuffer: optimize noSameBlockInflight check timing

* FDivSqrt: use hierarchy API to avoid dedup bug

* Dedup: use hartId from io port instead of core parameters

* Bump fudian

DCache timing problem has not been solved yet. DCache structure will be further changed.

* sbuffer: add extra perf counters

* sbuffer: optmize timeout replay check timing

* sbuffer: optmize do_uarch_drain check timing

Now we only compare merge entry's vtag, check will not start until
mergeIdx is generated by PriorityEncoder

* mem, lq: optmize writeback select logic timing

* dcache: replace missqueue reill req arbiter

* dcache: refactor missqueue entry select logic

* mem: add comments for lsq data

* dcache: give amo alu an extra cycle

* sbuffer: optmize sbuffer forward data read timing

* disable log as default
* code clean up