* ldu: optimize dcache hitvec wiring

In previous design, hitvec is generated in load s1, then send to dcache
and lsu (rs) side separately. As dcache and lsu (rs side) is far in real
chip, it caused severe wiring problem.

Now we generate 2 hitvec in parallel:

* hitvec 1 is generated near dcache.
To generate that signal, paddr from dtlb is sent to dcache in load_s1
to geerate hitvec. The hitvec is then sent to dcache to generate
data array read_way_en.

* hitvec 2 is generated near lsu and rs in load_s2, tag read result
from dcache, as well as coh_state, is sent to lsu in load_s1,
then it is used to calcuate hitvec in load_s2.  hitvec 2 is used
to generate hit/miss signal used by lsu.

It should fix the wiring problem caused by hitvec

* ldu: opt loadViolationQuery.resp.ready timing

An extra release addr register is added near lsu to speed up the
generation of loadViolationQuery.resp.ready

* l1tlb: replace NormalPage data module and add duplicate resp result

data module:
add BankedSyncDataMoudleWithDup data module:
divided the data array into banks and read as Async, bypass write data.
RegNext the data result * #banks. choose from the chosen data.

duplicate:
duplicate the chosen data and return to outside(tlb).
tlb return (ppn+perm) * #DUP to outside (for load unit only)

TODO: load unit use different tlb resp result to different module.
one for lsq, one for dcache.

* l1tlb: Fix wrong vidx_bypass logic after using duplicate data module

We use BankedSyncDataMoudleWithDup instead of SyncDataModuleTemplate,
whose write ports are not Vec.
Co-authored-by: NWilliam Wang <zeweiwang@outlook.com>
Co-authored-by: NZhangZifei <1773908404@qq.com>
Co-authored-by: Ngood-circle <fenghaoyuan19@mails.ucas.ac.cn>

This commit optimizes the timing of load-load forwarding by making
it speculatively issue requests to TLB/dcache.

When load_s0 does not have a valid instruction and load_s3 writes
a valid instruction back, we speculatively bypass the writeback
data to load_s0 and assume there will be a pointer chasing instruction
following it. A pointer chasing instruction has a base address that
comes from a previous instruction with a small offset. To avoid timing
issues, now only when the offset does not change the cache set index,
we reduce its latency by speculatively issuing it.

each tlb's port can be configured to be block or non-blocked.
For blocked port, there will be a req miss slot stored in tlb, but belong to
core pipeline, which means only core pipeline flush will invalid them.

For another, itlb also use PTW Filter but with only 4 entries.
Last, keep svinval extension as usual, still work.


* tlb: add blocked-tlb support, miss frontend changes

* tlb: remove tlb's sameCycle support, result will return at next cycle

* tlb: remove param ShouldBlock, move block method into TLB module

* tlb: fix handle_block's miss_req logic

* mmu.filter: change filter's req.ready to canEnqueue

when filter can't let all the req enqueue, set the req.ready to false.
canEnqueue after filtering has long latency, so we use **_fake
without filtering, but the filter will still receive the reqs if
it can(after filtering).

* mmu.tlb: change name from BTlbPtwIO to VectorTlbPtwIO

* mmu: replace itlb's repeater to filter&repeaternb

* mmu.tlb: add TlbStorageWrapper to make TLB cleaner

more: BlockTlbRequestorIO is same with TlbRequestorIO, rm it

* mmu.tlb: rm unused param in function r_req_apply, fix syntax bug

* [WIP]icache: itlb usage from non-blocked to blocked

* mmu.tlb: change parameter NBWidth to Seq of boolean

* icache.mainpipe: fix itlb's resp.ready, not always true

* mmu.tlb: add kill sigal to blocked req that needs sync but fail

in frontend, icache,itlb,next pipe may not able to sync.
blocked tlb will store miss req ang blocks req, which makes itlb
couldn't work. So add kill logic to let itlb not to store reqs.

One more thing: fix icache's blocked tlb handling logic

* icache.mainpipe: fix tlb's ready_recv logic

icache mainpipe has two ports, but these two ports may not valid
all the same time. So add new signals tlb_need_recv to record whether
stage s1 should wait for the tlb.

* tlb: when flush, just set resp.valid and pf, pf for don't use it

* tlb: flush should concern satp.changed(for blocked io now)

* mmu.tlb: add new flush that doesn't flush reqs

Sfence.vma will flush inflight reqs and flushPipe
But some other sfence(svinval...) will not. So add new flush to
distinguish these two kinds of sfence signal

morw: forget to assign resp result when ptw back, fix it

* mmu.tlb: beautify miss_req_v and miss_v relative logic

* mmu.tlb: fix bug, when ptw back and bypass, concern level to genPPN

bug: when ptw back and bypass, forgot to concern level(1GB/2MB/4KB)
when genPPN.

by the way: some funtions need ": Unit = ", add it.

* mmu.filter: fix bug of canEnqueue, mixed with tlb_req and tlb.req

* icache.mainpipe: fix bug of tlbExcp's usage, & with tlb_need_back

Icache's mainpipe has two ports, but may only port 0 is valid.
When a port is invalid, the tlbexcp should be false.(Actually, should
be ignored).
So & tlb_need_back to fix this bug.

* sfence: instr in svinval ext will also flush pipe

A difficult problem to handle:
Sfence and Svinval will flush MMU, but only Sfence(some svinval)
  will flush pipe. For itlb that some requestors are blocked and
  icache doesn't recv flush for simplicity, itlb's blocked ptw req
  should not be flushed.
It's a huge problem for MMU to handle for good or bad solutions. But
  svinval is seldom used, so disable it's effiency.

* mmu: add parameter to control mmu's sfence delay latency

Difficult problem:
  itlb's blocked req should not be abandoned, but sfence will flush
  all infight reqs. when itlb and itlb repeater's delay is not same(itlb
  is flushed, two cycles later, itlb repeater is flushed, then itlb's
  ptw req after flushing will be also flushed sliently.
So add one parameter to control the flush delay to be the same.

* mmu.tlb: fix bug of csr.priv's delay & sfence valid when req fire

1. csr.priv's delay
csr.priv should not be delayed, csr.satp should be delayed.
for excep/intr will change csr.priv, which will be changed at one
instruction's (commit?). but csrrw satp will not, so satp has more
cycles to delay.
2. sfence
when sfence valid but blocked req fire, resp should still fire.
3. satp in TlbCsrBundle
let high bits of satp.ppn to be 0.U

* tlb&icache.mainpipe: rm commented codes

* mmu: move method genPPN to entry bundle

* l1tlb: divide l1tlb flush into flush_mmu and flush_pipe

Problem:
For l1tlb, there are blocked and non-blocked req ports.
For blocked ports, there are req slots to store missed reqs.
Some mmu flush like Sfence should not flush miss slots for outside
may still need get tlb resp, no matter wrong and correct resp.
For example. sfence will flush mmu and flush pipe, but won't flush
reqs inside icache, which waiting for tlb resp.
For example, svinval instr will flush mmu, but not flush pipe. so
tlb should return correct resp, althrough the ptw req is flushed
when tlb miss.

Solution:
divide l1tlb flush into flush_mmu and flush_pipe.
The req slot is considered to be a part of core pipeline and should
only be flushed by flush_pipe.
flush_mmu will flush mmu entries and inflight ptw reqs.
When miss but sfence flushed its ptw req, re-send.

* l1tlb: code clean, correct comments and rm unused codes

* l2tlb: divide filterSize into ifiterSize and dfilterSize

* l2tlb: prefetch req won't enter miss queue. Rename MSHR to missqueue

* l1tlb: when disable vm, ptw back should not bypass tlb and should let miss req go ahead

* add SRAM ready (resetfinish) condition for *Array (metaArray/dataArray)
req.ready

* fix some typos
Signed-off-by: Ncuishuang <imcusg@gmail.com>

* ITLB <timing>: delay miss and flush req for ITLB

* add 2 ILTB requestor and delete tlb_arb

* Bump huancun

* ICacheMainPipe <bug-fix>: fix slot invalid condition

* ITLB <timing>: add port to 6

* ICacheMainPipe <bug-fix>: stop pipe when tlb miss

* ICacheMainPipe <bug-fix>: fix illegal flush
Co-authored-by: NLinJiawei <linjiawei20s@ict.ac.cn>

* ICache <timing>: move parity decode to pipe

* ICacheMainPipe <timing>: remove parity af

* ReplacePipe <timing>: delay error generating

* ReplacePipe: block miss until get ReleaseAck

* IPrefetch: cancle prefetch req when meet MSHR

* Fetch <perf>: add fetch bubble performance counters

* IFU <timing>: f2_data select signal optimization

* ICacheMainPipe <timing>: latch fetch req when tlb miss

* Frontend <timing>: add additional PMP checker

* Ftq <timing>: delete flush condition for prefetch.req

* ICacheMainPipe <timing>: move hit state change to s2

* ICache <bug-fix> delete PMP check assertion

* ICache <bug-fix> fix parity error condition

* ICacheMainPipe <bug-fix>: fix tlb resp condition

* when TLB req has been latched into tlb_slot, the
tlb_all_resp condition, which affects s0_fire should
depend on the slot result.

* when TLB req has been latched into tlb_slot, the
tlb_all_resp condition, which affects s0_fire should
depend on the slot result.

* mem: fix error csr update

* dcache: l2 error will now trigger atom error

* chore: fix cache error debug decoder

* mem: split L1CacheErrorInfo and L1BusErrorUnitInfo

* fix performance counter in ICacheMainpipe

* IPrefetch: add prefetch address merge and counter

* Add Prefetch and Parity enable register for ICache

* Add ICache parity enable control for pipe

* dcache: add source info in L1CacheErrorInfo

* ICache: fix valid signal and add source/opType

* dcache: fix bug in ecc error

* mem,csr: send full L1CacheErrorInfo to CSR

* icache: provide cache error info for CSR

* dcache: force resp hit if tag ecc error happens

* mem: reorg l1 cache error report path

Now dcache tag error will force trigger a hit

* dcache: fix readline ecc check error

* dcache: mainpipe will not be influenced by tag error

* dcache: fix data ecc check error

* dcache: if coh state is Nothing, do not raise error
Co-authored-by: Nzhanglinjuan <zhanglinjuan20s@ict.ac.cn>
Co-authored-by: NJinYue <jinyue20s@ict.ac.cn>

* IPrefetch: fix prefetchPtr stop problem

* This problem happens because prefetchPtr still exits when close IPrefetch

* Fix PMP req port still be occupied even when ICache miss

* Shut down IPrefetch

* IPrefetch: fix Hint not set PreferCache bit

* bump HuanCun

* IPrefetch: fix prefetchPtr stop problem

* This problem happens because prefetchPtr still exits when close IPrefetch

* Fix PMP req port still be occupied even when ICache miss

* Shut down IPrefetch

* IPrefetch: fix Hint not set PreferCache bit

* bump HuanCun

* This problem happens because prefetchPtr still exits when close IPrefetch

* Fix PMP req port still be occupied even when ICache miss

* Add Naive Instruction Prefetch

* Add instruction prefetch module in ICache

* send Hint to L2 (prefetched data stores in L2)

* Ftq: add prefetchPtr and prefetch interface

* Fix IPrefetch PMP Port preempting problem

* Fix merge conflict

* ICache: raise access fault when L2 send corrupt

* ICache: add ECC error connection

* chores: add comments and code clean-up

* ICache: raise AF when Meta/Data Parity wrong

* Update Frontend.scala

* ICache: add ReplacePipe for Probe & Release

* remove ProbeUnit

* Probe & Release enter ReplacePipe

* fix bugs when running Linux on MinimalConfig

* TODO: set conflict for ReplacePipe

* ICache: fix ReplacePipe invalid write bug

* chores: code clean up

* IFU: optimize timing

* PreDecode: separate into 2 module for timing optimization

* IBuffer: add enqEnable to replace valid for timing

* IFU/ITLB: optimize timing

* IFU: calculate cut_ptr in f1

* TLB: send req in f1 and wait resp in f2

* ICacheMainPipe: add tlb miss logic in s0

* Optimize IFU timing

* IFU: fix lastHalfRVI bug

* IFU: fix performance bug

* IFU: optimize MMIO commit timing

* IFU: optmize trigger timing and add frontendTrigger

* fix compile error

* IFU: fix mmio stuck bug

* use toMeta.fire() will cause data miss match when toMeta.valid :=
req.valid

* ICache: add ReplacePipe for Probe & Release

* remove ProbeUnit

* Probe & Release enter ReplacePipe

* fix bugs when running Linux on MinimalConfig

* TODO: set conflict for ReplacePipe

* ICache: add Block logic for ReplacePipe

* ReplacePipe: change probe assert condition

* support Probe NToN (Probe not hit in ICache)

* ICache: fix a bug in meta_write_arb

* ICache: metaArray & dataArray use bank interleave

* ICache: add bank interleave

* ICache: add parity check for meta and data arrays

* IFU: fix bug in secondary miss

* secondary miss doesn't send miss request to miss queue

* ICache: write back cancled miss request

* ICacheMissEntry: add second miss merge

* deal with situations that this entry has been flushed, and the next miss req just
requests the same cachline.

* ICache: add acquireBlock and GrantAck support

* refact: move icache modules to frontend modules

* ICache: add release surport and meta coh

* ICache: change Get to AcquireBlock for A channel

* rebuild: change ICachePara package for other file

* ICache: add tilelogger for L1I

* ICahce: add ProbeQueue and Probe Process Unit

* ICache: add support for ProbeData

* ICahceParameter: change tag code to ECC

* ICahce: fix bugs in connect and ProbeUnit

* metaArray/dataArray responses are not connected

* ProbeUnit use reg so data and req are not synchronized

* RealeaseUnit: write back mata when voluntary

* Add ICache CacheInstruction

* move ICache to xiangshan.frontend.icache._

* ICache: add CacheOpDecoder

* change ICacheMissQueue to ICacheMissUnit

* ProbeUnit: fix meta data not latch bug

* IFU: delete releaseSlot and add missSlot

* IFU: fix bugs in missSlot state machine

* IFU: fix some bugs in miss Slot

* IFU: move out fetch to ICache Array logic

* ReleaseUnit: delete release write logic

* MissUnit: send Release to ReleaseUnit after GAck

* ICacheMainPipe: add mainpipe and stop logic

* when f3_ready is low, stop the pipeline

* IFU: move tlb and array access to mainpipe

* Modify Frontend and ICache top for mainpipe

* ReleaseUnit: add probe merge status register

* ICache: add victim info and release in mainpipe

* ICahche: add set-conflict logic

* Release: do not invalid meta after sending release

* bump Huancun: fix probe problem

* bump huancun for MinimalConfig combinational loop

* ICache: add LICENSE for new files

* Chore: remove debug code and add perf counter

* Bump huancun for bug fix

* Bump HuanCun for alias bug

* ICache: add dirty state for CliendMeta