Since you've built a 4-bit ALU, I'm going to assume you're interested in building a 4-bit computer. It will be almost half the work of building a 8-bit one, since the busses will be half as wide.
The following diagram is from a paper titled "A Simple and Affordable TTL Processor for the Classroom". It describes in quite a bit of detail the architecture of a 4-bit computer called CHUMP ("Cheap Homebrew Understandable Minimal Processor").
It is a Harvard architecture, meaning the program and data are in separate memories (program ROM and RAM in the diagram). It uses a single accumulator register (Accum), rather than the register arrays typically found in microcontrollers today.
PC is program counter which holds the address of the next instruction in the program ROM. The control ROM is used to decode the instructions. Each CHUMP instruction uses a 4-bit opcode (maximum of 16 instructions) and a 4-bit operand. This means it can load only one nibble of immediate data (0-F), address 16 RAM locations, perhaps 16 I/O ports, and be limited to 16 instructions. But these limitations will make it easier to build. You'll still need 64 FF's for your RAM.
You pretty much need to have all of the blocks shown above. If you are going to be making very small programs, you could use some sort of 16 x 4 plugboard instead of using EPROMs. Perhaps you could use a solderless breadboard. This one with 30 rows could store 14 instructions (7x4 on each side). Storing programs in RAM ("Von Neumann architecture") would be problematic since you would have to have a way of loading the program into RAM anyway.
Here are how the busses (all 4-bit) are wired. The output of the PC selects the address in program ROM. The input to the PC comes from the instruction operand in ROM (Sel Mux=0). This same path can be used to load an address from the instruction operand into the Addr latch, which selects a nibble from RAM. The RAM is read into the ALU from RAM with Sel Mux=1, or from the Program ROM (Sel Mux=0) as a constant literal. The Accum (accumulator) can be go to either the other side of the ALU, or be written into RAM. There is also a path to load either the PC or Addr latch from RAM, but I don't know if that's used.
Instructions are executed in several cycles, the advancing of the cycles is done using the system clock.
- First the first nibble of the instruction containing the opcode is
fetched from the program ROM. (As mentioned before, each instruction
is two nibbles long, a 4-bit opcode and a 4-bit operand.)
- Next decode the opcode (shown as control ROM in the block diagram;
this is misleading, you can do it just with logic).
- Read the second nibble of the instruction, which will be either an
address in program ROM, a constant (literal) in program ROM, or an
address in RAM.
- Execute the decoded instruction.
If the PC is not loaded from the instruction operand (jump instruction), then it is incremented after every instruction.
I suggest running your clock very slow, maybe only 1 MHz or so, so you can easily trace the operation using an oscilloscope. You probably want to add a single-step feature also, which executes one instruction per button push. Lots of blinking LEDs connected to the PC, Addr latch, and accumulator are good too.
Here is a page which has links to six other 4-bit computer pages, many with schematics. And here are two more, which weren't on the list above.