* freelist & refcounter: implement arch states

* walk: restore and walk again when redirecting

* ROB: optimize invalidation of `valid`

Now we update data field (fwd data, uop) in load queue when load_s2
is valid. It will help to on lq wen fanout problem.

State flags will be treated differently. They are still updated
accurately according to loadIn.valid
Co-authored-by: NWilliam Wang <zeweiwang@outlook.com>

Move imm addition to stage 0.

each tlb's port can be configured to be block or non-blocked.
For blocked port, there will be a req miss slot stored in tlb, but belong to
core pipeline, which means only core pipeline flush will invalid them.

For another, itlb also use PTW Filter but with only 4 entries.
Last, keep svinval extension as usual, still work.


* tlb: add blocked-tlb support, miss frontend changes

* tlb: remove tlb's sameCycle support, result will return at next cycle

* tlb: remove param ShouldBlock, move block method into TLB module

* tlb: fix handle_block's miss_req logic

* mmu.filter: change filter's req.ready to canEnqueue

when filter can't let all the req enqueue, set the req.ready to false.
canEnqueue after filtering has long latency, so we use **_fake
without filtering, but the filter will still receive the reqs if
it can(after filtering).

* mmu.tlb: change name from BTlbPtwIO to VectorTlbPtwIO

* mmu: replace itlb's repeater to filter&repeaternb

* mmu.tlb: add TlbStorageWrapper to make TLB cleaner

more: BlockTlbRequestorIO is same with TlbRequestorIO, rm it

* mmu.tlb: rm unused param in function r_req_apply, fix syntax bug

* [WIP]icache: itlb usage from non-blocked to blocked

* mmu.tlb: change parameter NBWidth to Seq of boolean

* icache.mainpipe: fix itlb's resp.ready, not always true

* mmu.tlb: add kill sigal to blocked req that needs sync but fail

in frontend, icache,itlb,next pipe may not able to sync.
blocked tlb will store miss req ang blocks req, which makes itlb
couldn't work. So add kill logic to let itlb not to store reqs.

One more thing: fix icache's blocked tlb handling logic

* icache.mainpipe: fix tlb's ready_recv logic

icache mainpipe has two ports, but these two ports may not valid
all the same time. So add new signals tlb_need_recv to record whether
stage s1 should wait for the tlb.

* tlb: when flush, just set resp.valid and pf, pf for don't use it

* tlb: flush should concern satp.changed(for blocked io now)

* mmu.tlb: add new flush that doesn't flush reqs

Sfence.vma will flush inflight reqs and flushPipe
But some other sfence(svinval...) will not. So add new flush to
distinguish these two kinds of sfence signal

morw: forget to assign resp result when ptw back, fix it

* mmu.tlb: beautify miss_req_v and miss_v relative logic

* mmu.tlb: fix bug, when ptw back and bypass, concern level to genPPN

bug: when ptw back and bypass, forgot to concern level(1GB/2MB/4KB)
when genPPN.

by the way: some funtions need ": Unit = ", add it.

* mmu.filter: fix bug of canEnqueue, mixed with tlb_req and tlb.req

* icache.mainpipe: fix bug of tlbExcp's usage, & with tlb_need_back

Icache's mainpipe has two ports, but may only port 0 is valid.
When a port is invalid, the tlbexcp should be false.(Actually, should
be ignored).
So & tlb_need_back to fix this bug.

* sfence: instr in svinval ext will also flush pipe

A difficult problem to handle:
Sfence and Svinval will flush MMU, but only Sfence(some svinval)
  will flush pipe. For itlb that some requestors are blocked and
  icache doesn't recv flush for simplicity, itlb's blocked ptw req
  should not be flushed.
It's a huge problem for MMU to handle for good or bad solutions. But
  svinval is seldom used, so disable it's effiency.

* mmu: add parameter to control mmu's sfence delay latency

Difficult problem:
  itlb's blocked req should not be abandoned, but sfence will flush
  all infight reqs. when itlb and itlb repeater's delay is not same(itlb
  is flushed, two cycles later, itlb repeater is flushed, then itlb's
  ptw req after flushing will be also flushed sliently.
So add one parameter to control the flush delay to be the same.

* mmu.tlb: fix bug of csr.priv's delay & sfence valid when req fire

1. csr.priv's delay
csr.priv should not be delayed, csr.satp should be delayed.
for excep/intr will change csr.priv, which will be changed at one
instruction's (commit?). but csrrw satp will not, so satp has more
cycles to delay.
2. sfence
when sfence valid but blocked req fire, resp should still fire.
3. satp in TlbCsrBundle
let high bits of satp.ppn to be 0.U

* tlb&icache.mainpipe: rm commented codes

* mmu: move method genPPN to entry bundle

* l1tlb: divide l1tlb flush into flush_mmu and flush_pipe

Problem:
For l1tlb, there are blocked and non-blocked req ports.
For blocked ports, there are req slots to store missed reqs.
Some mmu flush like Sfence should not flush miss slots for outside
may still need get tlb resp, no matter wrong and correct resp.
For example. sfence will flush mmu and flush pipe, but won't flush
reqs inside icache, which waiting for tlb resp.
For example, svinval instr will flush mmu, but not flush pipe. so
tlb should return correct resp, althrough the ptw req is flushed
when tlb miss.

Solution:
divide l1tlb flush into flush_mmu and flush_pipe.
The req slot is considered to be a part of core pipeline and should
only be flushed by flush_pipe.
flush_mmu will flush mmu entries and inflight ptw reqs.
When miss but sfence flushed its ptw req, re-send.

* l1tlb: code clean, correct comments and rm unused codes

* l2tlb: divide filterSize into ifiterSize and dfilterSize

* l2tlb: prefetch req won't enter miss queue. Rename MSHR to missqueue

* l1tlb: when disable vm, ptw back should not bypass tlb and should let miss req go ahead

* ctrl: copy dispatch2 to avoid cross-module loops

This commit makes copies of dispatch2 in CtrlBlock to avoid long
cross-module timing loop paths. Should be good for timing.

* dpq: re-write queue read logic

This commit adds a Reg-Vec to store the queue read data. Since
most queues read at most the current numRead and the next numRead
entries, the read timing can be optimized by reading the data one
cycle earlier.

This commit re-pipelines ECC check logic in data cache and exception generate logic for better timing.
Now ecc error is checked 1 cycle after reading result from data sram. An extra cycle is added for load
writeback to ROB.

Future work: move the pipeline to https://github.com/OpenXiangShan/XiangShan/blob/master/src/main/scala/xiangshan/backend/CtrlBlock.scala#L266-L277, which add a regnext.

* dcache: repipeline ecc check logic for timing

* chore: fix normal loadAccessFault logic

* wbu: delay load unit wb for 1 cycle

* dcache: add 1 extra cycle for beu error report

* feat: parameterize load/store pipeline, etc.

* fix: use LoadPipelineWidth rather than LoadQueueSize

* fix: parameterize `rdataPtrExtNext`

* SBuffer: fix idx update logic

* atomic: parameterize atomic logic in `MemBlock`

* StoreQueue: update allow enque requirement

* feat: support one load/store pipeline

* feat: parameterize `EnsbufferWidth`

* chore: resharp codes for better generated name

This commit fixes the implementation of WFI. The WFI instruction
waits in the ROB until an interrupt might need servicing.

According to the RISC-V manual, the WFI must be unaffected by the
global interrupt bits in `mstatus` and the delegation register
`mideleg`.

The RISC-V WFI instruction is previously decoded as NOP. This commit
adds support for the real wait-for-interrupt (WFI).

We add a state_wfi FSM in the ROB. After WFI leaves the ROB, the next
instruction will wait in the ROB until an interrupt.

old missqueue: cache req miss slot and mem access-er
Problem: these two func are totally different, make mq hard to handle in a single select policy.
Solution: divide these two funciton into two module.
  new MissQueue: only hold reqs that page cache miss and need re-req cache， a simple flushable queue
  llptw: Last level ptw, only access ptes, priorityMux queue

* mmu: rename PTW.scala to L2TLB.scala

* mmu: rename PTW to L2TLB

* mmu: rename PtwFsm to PTW

* mmu.l2tlb: divide missqueue into 'missqueue' and llptw

old missqueue: cache req miss slot and mem access-er
Problem: these two func are totally different, make mq hard to handle
  in single select policy.
Solution: divide these two funciton into two module.
  new MissQueue: only hold reqs that page cache miss and new re-req
  cache
  llptw: Last level ptw, only access ptes

* mmu.l2tlb: syntax bug that misses io assign

* mmu.l2tlb: fix bug that mistakes ptw's block signal

* Adjusted reset signals

* Support reset tree

* mem: fix error csr update

* dcache: l2 error will now trigger atom error

* chore: fix cache error debug decoder

* mem: split L1CacheErrorInfo and L1BusErrorUnitInfo

This commit adds an LsqEnqCtrl module to add one more clock cycle
between dispatch and load/store queue.

LsqEnqCtrl maintains the lqEnqPtr/sqEnqPtr and lqCounter/sqCounter.
They are used to determine whether load/store queue can accept new
instructions. After that, instructions are sent to load/store queue.
This module decouples queue allocation and real enqueue.

Besides, uop storage in load/store queue are optimized. In dispatch,
only robIdx is required. Other information is naturally conveyed in
the pipeline and can be stored later in load/store queue if needed.
For example, exception vector, trigger, ftqIdx, pdest, etc are
unnecessary before the instruction leaves the load/store pipeline.

* mem,cacheop: fix read data writeback

* mem,cacheop: rename cacheop state bits

These bits are different from w_*, s_* bits in cache

* mem: enable icache op feedback

* icache: update cache op implementation

* chore: remove cache op logic from XSCore.scala

This commit optimizes the coding style and timing for hardware
performance counters.

By default, performance counters are RegNext(RegNext(_)).

This commit adds WritebackSink and WritebackSource parameters for
multiple modules. These traits hide implementation details from
other modules by defining IO-related functions in modules.

By using WritebackSink, ROB is able to choose the writeback sources.
Now fflags and exceptions are connected from exe units to reduce write
ports and optimize timing.

Further optimizations on write-back to RS and better coding style to
be added later.

* SoC: add axi4spliter

* pmp: add apply method to reduce loc

* pma: add PMA used in axi4's spliter

* Fix package import

* pma: re-write tl-pma, put tl-pma into AXI4Spliter

* pma: add memory mapped pma

* soc: rm dma port, rm axi4spliter, mv mmpma out of spliter

* Remove unused files

* update dma pma check port at SimTop.scala; update pll lock defalt value to 1
Co-authored-by: NZhangZifei <zhangzifei20z@ict.ac.cn>
Co-authored-by: Nrvcoresjw <shangjiawei@rvcore.com>

* FDivSqrt: use hierarchy API to avoid dedup bug

* Dedup: use hartId from io port instead of core parameters

* Bump fudian

* disable log as default
* code clean up

This fixes differences between the pingpong bits in ctrlblock and dispatch2.

* l2tlb: add repeater/filter's object apply method

* l2tlb: add one more cycle(repeater) between itlb and l2tlb

This commit changes when instructions enter load/store queue.
Now, at dispatch2, load/store instructions enter load/store queue.

* Add perf counters
* add reg from hpm counter source
* add print perfcounter enable

add mmu's asid support.
1. put asid inside sram (if the entry is sram), or it will take too many sources.
2. when sfence, just flush it all, don't care asid.
3. when hit check, check asid.
4. when asid changed, flush all the inflight ptw req for safety
5. simple asid unit test:
asid 1 write, asid 2 read and check, asid 2 write, asid 1 read and check. same va, different pa

* ASID: make satp's asid bits configurable to RW
* use AsidLength to control it

* ASID: implement asid refilling and hit checking
* TODO: sfence flush with asid

* ASID: implement sfence with asid
* TODO: extract asid from SRAMTemplate

* ASID: extract asid from SRAMTemplate
* all is down
* TODO: test

* fix write to asid

* Sfence: support rs2 of sfence and fix Fence Unit
* rs2 of Sfence should be Reg and pass it to Fence Unit
* judge the value of reg instead of the index in Fence Unit

* mmu: re-write asid

now, asid is stored inside sram, so sfence just flush it
it's a complex job to handle the problem that asid is changed but
no sfence.vma is executed. when asid is changed, all the inflight
mmu reqs are flushed but entries in storage is not influenced.
so the inflight reqs do not need to record asid, just use satp.asid

* tlb: fix bug of refill mask

* ci: add asid unit test
Co-authored-by: NZhangZifei <zhangzifei20z@ict.ac.cn>

This commit removes flush IO for every module. Flush now re-uses
redirect ports to flush the instructions.

预取时机：

    或者 发生miss时
    或者 发生hit，但是hit的entry是预取上来的
    当 页表2MB的level命中
    当 预取项不跨2MB项对应的4KB page frame

前面两个限制是为了限制预取的数量

后面两个限制是限制预取请求只会访问最后一级页表 -› 不占用FSM & （几乎）不会重新访问cache，造成卡死。

=============
some workloads: gcc(5.4%), wrf(13.6%),milc(9.2%)'s ipc increase.
some workloads decrease: namd(-2.5%).
but l2tlb's perf counters are better.
So I think it is worthy to adding the simple next-line prefetch.

The workloads are of ci and in cold-start state, so prefetch may seems to be much better than it should be.
But l2tlb's memory access ability is much better than what it needs, so the prefetch can be added.
=============

* mmu.l2tlb: add params filterSize

* mmu.l2tlb: add prefetch,dont work well

* mmu.l2tlb: add prefetch relative perf counter

* l2tlb: prefetch recv miss req and 'hit but pre-fetched' req

* l2tlb: fix some perf counter about prefetch

* l2tlb: prefetch not cross 2MB && not recv when 2MB level miss

* ci: when error, copy emu and SimTop.v to WAVE_HOME

* Temporarily disable TLMonitor

* Bump huancun (L2/L3 MSHR bug fix)

* Refactor Top

* Bump huancun

* alu: fix bug of rev8 & orc.b instruction
Co-authored-by: Zhangfw <471348957@qq.com>