Home computer BASIC performance tests, some benchmarks

When reading some web content on home computers comparisons, I started asking myself if there is a way of evaluating performances on a fair bases. This is rather difficult for 8 bit home computers, as each of them usually has a unique architecture. Different processors, different memory management schemes, different Operating Systems… all of this makes it extremely difficult to do benchmarks on these systems.

But, most of these systems do have one thing in common: a BASIC interpreter in ROM, available at start-up. Different home computers models may have different BASIC dialects, but very simple programs will run on all of them.

After reading this web page, I decided to test these programs on some home computers I own. I don’t have a big collection, so I couldn’t make a perfect choice. Still, in order not to make tests too long, I decided to only test five computers. Two 128 K machines, and three 64K ones:

  • Commodore 64;
  • Commodore 128;
  • Sinclair ZX Spectrum 128 +2;
  • MSX Panasonic CF 2700;
  • Amstrad CPC 464

All these tests have been performed on PAL machines.

The Commodore 64 is a computer from 1982 we definetely know about. It has a 6510 CPU running at only 0,98 MHz on PAL systems. 64K RAM of memory, only 38911 RAM bytes available for BASIC programming. Commodore 64 BASIC is actually a Microsoft BASIC implementation. 65xx family CPUs have a small instruction set and they have few registers. They enjoy fast memory access and for this reason they are “memory oriented” CPUs.

 

The Commodore 128 may be regarded as a powered-up version of the Commodore 64, from 1985.  It has 128K RAM and a 1Mhz/2MHz 8502 CPU, very similar to a 6510. This computer has two video chips: a variation of the C64 VIC-II chip, and a VDC, for 80 columns text. It also offers bitmap graphics, with a resolution of 640 x 200. Unfortunately, the VIC-II chip is not able to work properly when the CPU is at 2 MHz. So, when on C128 40 columns mode, the CPU must be forced to 1 MHz operation. 2 MHz can be activated on 40 columns mode actually, but no image will be shown. Still, as those benchmarks programs don’t use screen, blanking the screen may be tolerated. So, if you only have RF connection and you are not able to have the VDC video output, you can test those programs by adding a BASIC line with the FAST command at the very beginning of each program, and by adding another BASIC line with the command SLOW at the end of the programs. This way, screen will be blanked only during program execution. For our benchmark tests, the 2 MHz fast mode available on C128 80 columns mode will be used, so that programs will not need to be accomodated for the FAST/SLOW modes switching.

 

The Sinclair ZX Spectrum 128 +2 is a Spectrum version made by Amstrad in 1986. I choose this computer because is the only Spectrum I own at the moment. I had a 48K Spectrum but I sold it some time ago (and I do regret it). Being a Commodore 64 user, the Spectrum 128 keyboard is more confortable to me than the original 48K one. That helped me to type the benchmark programs with no problem. The +2 keyboard is excellent, despite the unusual position of the DELETE key. This computer offers 128K RAM of memory, and is powered by a Z80 CPU @ 3,54 MHz. This clock speed is much greater than the Commodore 64 one. The Z80 CPU is an 8 bit CPU, but it does have internal 16 bit registers. Furthermore, it has a bigger instructions set than 6502 CPUs and it is usually better for vectors calculations. Z80s are “register oriented” CPUs, as they have many registers in comparison to 6502s.

 

The PANASONIC CF 2700 is a MSX 1 machine. This is not very common, but this is the only MSX 1 machine I now have. By the way, I wanted another Z80 based machine to go along with the Spectrum, and here it is. It actually features a Z80 clone by NEC, but I don’t think it makes any difference. MSX machines have a vastly upgraded version of Microsoft BASIC, with many instructions.

 

The Amstrad CPC 464 was introduced in 1984 as a competitor to the Commodore 64 and ZX Spectrum. It was based on a Z80 CPU, running at 4.0 MHz, and on a Motorola 6845 video chip. Sound was offered by the AY-3-8910 chip. This model features 64 Kbytes of RAM. A BASIC interpreter made by Locomotive Software Ltd. is built in ROM.

The Commodore 64 has the older interpreter and the lowest CPU clock speed among these computers.

Now, let’s talk about those speed test programs.

With slight modifications of the generic versions, these benchmarks were made capable of running on all the machines with no custom code. This way, for each program the code is the same for every machine. For the Commodore 128, the FAST command was executed in direct mode before running the programs. Again, programs are taken from this page.

Benchmark 1

20 FOR k=1 TO 1000
40 NEXT k

FOR NEXT cycles are really common on BASIC programs, so their execution speed gives a good idea of the interpreter overall performance.

 

Benchmark 2

20 LET k=0
30 LET k=k+1
50 IF k<1000 THEN GOTO 30

Instead of the more sophisticated FOR NEXT cycle, the iteration is performed by using a counter and a GOTO instruction. This is less efficient but allows to make an estimation on how fast branches are executed.

 

Benchmark 3

20 LET k=0
30 LET k=k+1
40 LET a=k/k*k+k-k
50 IF k<1000 THEN GOTO 30

Calculations have been added.This way, speed on simple computations and on access to variables may be evaluated.

 

Benchmark 4

20 LET k=0
30 LET k=k+1
40 LET a=k/2*3+4-5
50 IF k<1000 THEN GOTO 30

Very similar to the previous test, but variables have been replaced by constants on calculations. If variable access is slow, this test will run faster than benchmark 3.

 

Benchmark 5

20 LET k=0
30 LET k=k+1
40 LET a=k/2*3+4-5
45 GOSUB 700
50 IF k<1000 THEN GOTO 30

60 STOP

700 RETURN

A call to a subroutine has been introduced. Subroutines are frequently used in BASIC programming, so if subroutines calls are slow, the whole program will be slow as well. I added a line: 60 STOP, so that no “RETURN WITHOUT GOSUB ERROR” will be shown.

 

Benchmark 6

20 LET k=0
25 DIM m(5)
30 LET k=k+1
40 LET a=k/2*3+4-5
45 GOSUB 700
46 FOR l=1 TO 5
48 NEXT l
50 IF k<1000 THEN GOTO 30

60 STOP
700 RETURN

This benchmark creates an array of five numeric elements. It also adds a FOR NEXT loop. The array will be needed on the next benchmark.

 

Benchmark 7

20 LET k=0
25 DIM m(5)
30 LET k=k+1
40 LET a=k/2*3+4-5
45 GOSUB 700
46 FOR l=1 TO 5
47 LET m(l)=a
48 NEXT l
50 IF k<1000 THEN GOTO 30

60 STOP
700 RETURN

Now, the array is used by assigning a value to each element of it. The slower this benchmark is in comparison to benchmark 6, the less efficient is array access.

 

Benchmark 8

20 LET k=0
25 DIM m(5)
30 LET k=k+1
40 LET a=k^2
45 LET b=LN(k)
47 LET c=SIN(k)
50 IF k<1000 THEN GOTO 30

This benchmark evaluates the efficiency of the implementation of mathematical functions. So, this program may give you an idea on how much a given interpreter is more or less adequate for creating efficient scientific programs.

 

THE RESULTS

The clock speed of the CPU doesn’t seem to always reflect the effective speed these programs will run at. Actually, the Commodore 64 computer, powered by a 0,98 MHz 6510 CPU, seems here to be faster than the MSX computer in average. It is also without doubt much faster on running those programs than Spectrum machines. Sinclair BASIC is actually the slowest on these performance tests. However, it should be noted that this interpreter was written for compactness rather than for speed.

However, the Amstrad CPC performs much better than the other Z80 based systems, sporting excellent execution times with these benchmarks, doing much better than the Commodore 64. The Amstrad machine also offers better performance than the Commodore 128 in 2MHz mode. On the first seven benchmarks, there’s no great difference between the Amstrad CPC and the Commodore 128. But, execution time of benchmark 8 on the Commodore 128 is twice as the time required by the Amstrad CPC. That means that Amstrad Locomotive BASIC provides much faster math routines than Commodore BASIC.

While doing tests on the Commodore 64, I intentionally kept inserted the Final Cartridge III I routinely use. This cartridge may slow down BASIC a bit, but as it is common practice to use cartridge tools on the C64 (for instance, to gain faster and easier disk access) I decided to keep it active while doing benchmarks. It shouldn’t make a great difference, so I don’t think this is unfair to the Commodore 64.

The MSX BASIC doesn’t seem very fast, although perfomances are much better than Sinclair BASIC. But, its performances are inferior in comparison to those of the Commodore 128 in 2MHz mode and of the Amstrad CPC 464. Indeed, some benchmarks on the MSX machine are faster than the ones on the Commodore 64, but benchmark 8 is much more faster on the Commodore 64.

It seems that, at least with some interpreters, 65xx CPUs have better average performances than Z80s CPUs. This may be due to the fact that executing BASIC programs requires many variables involved. So, maybe Z80 registers get saturated and fast memory access of 65xx CPUs allows for better performances on those particular situations.

Nevertheless, the Amstrad CPC (with a 4 MHz Z80 CPU) offers excellent performances on these benchmarks, turning out to be the fastest machine here (there are also other benchmarks were the Amstrad CPC offers very good performances). So, maybe it is just that MSX BASIC and Sinclair BASIC are both not well optimized for speed performances.

I do own a Commodore 16 and a Commodore VIC 20, but I decided not to test them, in order not to have too much Commodore machines on the “competition”.

Those tests are pretty aleatory. I think they give a good idea of the performances of the BASIC interpreters, but they cannot be regarded as fully representative of the overall performances of a given machine. Benchmarks written on machine language may give different results. But, BASIC allows to test the very same code on each machine, something Machine Language doesn’t make possible.

Futhermore, those programs are short. Each interpreter should be tested on very long programs as well. Accuracy on calculations that each interpreter offers should also be taken into account.

All that said, here are the results. I measured the time manually for each program execution, without using internal clocks on the computers that have it. This way, the same method of measurement has been used for all the machines.

 

SPECTRUM 128 +2 (grey)

Bench01:          6,2   sec
Bench02:         12,8   sec
Bench03:         26,2   sec
Bench04:         25,5   sec
Bench05:         32,0   sec
Bench06:   1 min 12,0   sec
Bench07:   1 min 43,0   sec
Bench08:   4 min  6,0   sec

 

MSX PANASONIC CF 2700

Bench01:          2,0   sec
Bench02:          6,0   sec
Bench03:         17,0   sec
Bench04:         18,5   sec
Bench05:         19,5   sec
Bench06:         42,0   sec
Bench07:         45,0   sec
Bench08:   3 min 33,0   sec

 

COMMODORE 64

Bench01:          1,7   sec
Bench02:         11,0   sec
Bench03:         21,0   sec
Bench04:         23,0   sec
Bench05:         24,8   sec
Bench06:         36,7   sec
Bench07:         58,9   sec
Bench08:   2 min  0,0   sec

 

COMMODORE 128 (80 COL, 2MHz mode)

Bench01:           1,2  sec
Bench02:           5,7  sec
Bench03:          10,7  sec
Bench04:          11,2  sec
Bench05:          12,5  sec
Bench06:          20,0  sec
Bench07:          32,5  sec
Bench08:    1 min  1,0  sec

 

AMSTRAD CPC 464

Bench01:           1,2  sec
Bench02:           3,7  sec
Bench03:           9,4  sec
Bench04:           9,9  sec
Bench05:          10,5  sec
Bench06:          19,4  sec
Bench07:          31,0  sec
Bench08:          34,6  sec

 

It should be noted that benchmark 7 on the MSX computer is noticeably faster than the same benchmark on the Commodore 64. Also, there is little difference on execution times between benchmarks 6 and 7 on the MSX machine. This may indicate a fast array access on MSX BASIC and a slower array access on Commodore BASIC. Even on the Commodore 128 and on the Amstrad CPC 464 you may notice a remarkable difference in execution time between benchmarks 6 and 7. This difference is much more noticeable on Sinclair BASIC, which seems to also provide the slowest array access among all of the tested machines.

Precedente A review of the Commodore 64 computer: overall features, memory management, applications Successivo Garbage collection on Commodore 64 BASIC, how to handle it

Lascia un commento

*