2020-07-30 20:54 UTC
The world is in a weird state right now. I hope you are all doing well. For me this stay-at-home and work-from-home situation has at least given me some time to re-evaluate my shortwave transceiver project. To be honest I wasn't happy with the performance I got out of my superheterodyne design. It got more and more complex in order to achieve good performance across the intended shortwave range. It got complex to the point where I deviated far away from the goal of making a simple, understandable and yet decent performing transceiver out of common off-the-shelf parts. Either way I turned and twisted my design it ended up being too much of a compromise for my liking. This was not what I had in mind at all. Let's just say there is a reason good performing transceivers of this kind are expensive. Dealing with multi stage mixing and filtering is powerful but doing it right takes a lot of careful calculations and filtering. Also every stage of the transceiver also impose a risk of degrading the signal (i.e. by introducing noise and intermodulation) making each stage even more complex and critical. To reach my goal I had to find another way.
2018-11-06 16:37 UTC
Yet again I feel like way to much time has passed since my latest project update. Or any update at all for that matter. This time I'm revisiting the world of early computing mass storage, namely, audio cassettes.
BackgroundMany of you probably have this love-hate relationship to data on audio cassettes. You know how unreliable and tediously slow it is but at the same time it has this cozy aura around it. I will never forget those times waiting half an hour for the program to load only to realize there was a load error and you had to do it all over again. Not to mention winding the tape back and forth according tho those hand written scrabbles and notes of various tape position values to find your precious data. Very hands on in a way modern computing rarely is today. I felt that my MC3 computer was missing out on a lot of fun so I decided to make an audio cassette interface for it.
Tapes and recordersData on audio tapes is nothing more than tones that the computer can generate and interpret when played back. The audio must be constrained to the limitations of the audio cassette and the recorder used. There are many variants here but I will focus on the, probably most common, Compact Audio Cassette format. There are different types of magnetic tapes in those cassettes that yields different frequency responses. Generally you have a usable response up to around 15kHz using a "normal" iron oxide tape. Chrome or metal tapes can have a frequency response of up to 20kHz when using good tapes and good recorder. Also cassettes made for shorter recording times are usually made out of thicker material thus making them more robust and perform slightly better. My goal is to store data using normal tapes together with a Panasonic RQ-2102 recorder. The RQ-2102 is a quite common recorder that has been in production for a long time and has historically been popular among computer users. It was even the recommended recorder by Woz himself for use together with the Apple I. I think I have the latest incarnation of the RQ-2102. It has changed appearance slightly during the years but the function and performance has been more or less the same as far as I know. Unfortunately the RQ-2102 is not equipped with a remote jack for controlling the motor from the computer which would be a nice feature to have but for my purposes it will do just fine.
HardwareThis is actually a very simple I/O card for my MC3 computer. There is a trade-off to be made here between hardware complexity, performance and flexibility. A complex interface that do most of the encoding and decoding in hardware will perform good but is most likely not very flexible. A simple interface, like the one I have here, do the encoding and decoding in software and because of that it is flexible and could, in theory, handle different encodings by using different software algorithms. Most home computers from the cassette era used simple bit-banged interfaces, just like this one, and software based encodings. That made it possible for users to develop custom loaders for example and exchange cassettes between different systems. My cassette interface is no more than a software controlled single bit I/O port and single bit ADC/DAC and the whole circuit fit on a small MC3 I/O board. The digital part of this board is not very exciting. It's just a '244 buffer (U3) for reading and a '374 latch (U4) for writing. That makes up 7 more bits than actually needed but they may come of use some day, who knows. By writing and toggling bit zero, an audio square wave can be generated and recorded on to the tape. A simple resistor divider brings the output level from the '374 down to microphone level for the tape recorder's input. Some experimenting may be required here to find a good level but in the schematic are the values that I found worked well for my RQ-2102. Reading back the waveform reliably from the recorder is a little bit more complicated. There is no guarantee that the shape and amplitude of the waveform is maintained during recording and playback. Therefore a bit of analog trickery is needed here to shape the incoming signal back into a logic level square wave that can be sampled by the '244 buffer input. I do this by using an LM358 op-amp (U2) connected as a comparator. The incoming signal is passed through a capacitor and biased at Vcc/2. The negative input of the op-amp is fixed at the same Vcc/2 and the positive input is fed the biased input from the recorder. This way, zero-crossings in the incoming signal are detected and converted to a logic level square wave that the '244 can pick up reliably. Because of the bit-banging and timing critical nature of this interface I also added an LED to show activity on the board since the CPU will most likely be too busy to write anything useful to the console during operation. An LED is a crude way to at least give some clue about what is going on.
Initial testingFirst off let me just explain why I choose to use bit zero for the output and bit seven for the input. The reasons for this is on the software side. Access to the interface must be made fast and efficient. Using bit zero for the output makes sense since bit zero of a register can quickly be toggled using a DEC or INC instruction. Similarly using bit seven for the input means that conditional branches can be made based on the bit value by using BMI and BPL instructions. These two tricks may seem insignificant but it can be a big advantage when timing is critical. To verify that the interface was in fact working I made a simple loop to mirror the input to the output.
casport equ $8060 org $0100 loop ldaa casport rola rola staa casport bra loopBit seven is read and shifted around to bit zero and written back in an endless loop. By using a signal generator and oscilloscope I was able to verify that the circuit was working as intended.
Storing bitsWith the knowledge that the hardware side of the project was working I began tinkering with a simple low level format for data on tapes. Instead of trying to develop software to support an existing format I decided to start from scratch and make something of my own to cover the basics and to verify the concept. It's important to keep the signal moving in order to lock on to the data stream. Never let it be static for too long. I decided to go for an encoding scheme that uses one cycle per bit. There are many ways to accomplish this but I ended up implementing these three variants. 1. Frequency shifting One cycle of a high frequency represents "0" and a single cycle of low frequency represents "1". This effectively makes ones and zeros have different length. That means the average bit rate will vary depending on the distribution of ones and zeros. 2. Varying pulse lengths Here the cycles has been evened out to be the same length. The short cycle has been made longer and the longer cycle has been made shorter. The relationship between the high part and the low part of the cycle determines if represents "1" or "0". This makes the bit rate constant regardless of the content. 3. Combination of 1 and 2 In this case only the length of one half of the cycle determines the bit content. The average bit rate should be the highest of the three but still vary with the content, albeit to a lesser extent. With some clever coding I hope to support all three variants in the same loading routine.
StructureIn order to load data back from tape we need a way to locate the first bit of the first byte of the data stream. The software would need a specific synchronization pattern to look for. I decided to make it simple and use a synchronization pattern consisting of a relatively long sequence of consecutive ones followed by a single zero to indicate end of sync. Data is then followed directly after the sync. I decided to have one start bit for each byte and then the bytes are clocked out with the least significant bit first. The start bit has the value zero since the sync pattern is made up from ones. That way it's possible to indicate end of data by sending another sync pattern of ones. The start bit also gives some small level of phase error checking. A better way may be to also add a parity bit to every byte but I will leave that for another time. This is the final structure I ended up with:
1. Synchronization pattern consisting of 2048 bits "1" and one single bit "0" 2. Start bit "0" 3. Eight data bits, LSB first 4. Repeat 2 & 3 until end of data 5. Synchronization pattern againBy ending with an identical synchronization pattern as the one in the beginning it's possible to add several consecutive data segments but for now I will start with only one to keep it simple.
Software implementation of SAVE routineThe save routine is pretty strait forward. I created separate bit delay subroutines to make it possible to easily change the bit rate. There is a subroutine for generating the sync pattern and a subroutine for sending a single byte. Saving data is done in a simple loop until all bytes are saved. The routine implements variant number three (see above) where the length of the high portion of the cycle determines the bit value. I set the timing in a way that makes the a "1" bit have twice the length as a "0" bit. The save routine asks for a start address and an end address to select what data is to be stored on tape. Below is an example printout when running the program. User input in blue text.
> j a100 BegAddr: 1000 EndAddr: 14ff Start tape and press enter <CR> OK >This is what it sounds like: audio clip (watch your playback volume!) Average data rate is around 255-300bps using this program. This could be improved with other timing parameters. Download my experimental tape save routine: - source - listing - s19
Software implementation of LOAD routineThe load routine is a little more complicated. Care must be taken here to accommodate different timings and possibly different tape speeds and bit rates. My goal was to make it robust enough to automatically detect the bit rate and load data without requiring the user to set any additional parameters. The bit rate is determined by averaging the cycle time of the synchronization pattern. The core part of the load routine is this subroutine, "cycle", that measures the time of the high portion of a cycle.
cycle: clra ldab casport bmi cycle3 ; if high on entry we are out of sync cycle0: inca beq cycle3 ; timeout ldab casport bpl cycle0 ; wait for positive flank clra cycle1: inca beq cycle3 ; timeout ldab casport bmi cycle1 ; wait for negative flank cycle3: rtsThe routine will return the time in A. When A is zero it indicates a timeout or that the routine was entered during a high portion of a cycle. This is used for error detection and to initially find and lock on to the synchronization pattern. The program begins by asking where in memory to store the data from the tape. When loading starts the program calls "cycle" repeatedly until a steady stream of cycles are found. That is detected when "cycle" returns a non zero value on every call. The first 256 steady cycles are used to determine the cycle time using a running average. Knowing the cycle time for the sync portion means we know the cycle time for a "1" bit and from that a threshold value is calculated that sits between a "1" cycle time and a "0" cycle time. Cycle time threshold value is the calculated using the formula: threshold = cycletime - cycletime/4. This is based on the assumption that the length of a "1" bit is twice the length of a "0" bit, thus this calculation places the threshold value right in between those values. After the cycle time has been calculated the program monitors the stream to make sure there is no interruption. If the bit stream is interrupted the synchronization will restart from the beginning again. As soon as a cycle time corresponding to a "0" bit is found it marks the end of the synchronization and data will follow. Data loading checks for a "0" start bit and receives eight bits and stores to memory. This is repeated until the start bit is not "0". This indicates either an error situation or that the ending synchronization has started. Below is an example printout when running the program. User input in blue text.
> j a100 DstAddr: 1000 Press enter and start tape <CR> Stop at 14FF >Download my experimental tape load routine: - source - listing - s19
Implementation for MCFS2Of course I couldn't resist making a few tools for my MCFS2 system as well. I created a simple checksum calculator also as a way to verify the tapes after load. casisave.asm - Cassette interface save (file to tape) - Syntax: casisave <filename> casiload.asm - Cassette interface load (tape to file) - Syntax: casiload <filename> chksum.asm - 16bit checksum calculator - Syntax: chksum <filename>
SummaryThese programs provide a relatively crude way of storing data on tapes but I think it's a good proof of concept. More work is needed to develop a proper format and error handling to make this usable in a real world scenario. Perhaps even a header with file meta data. That would be nice. This has been a fun project. It reminded me of time spent with the KIM-1 all those years ago. The reliability surprises me actually. No failed loads at all so far.
2018-02-17 15:22 UTC
I know these updates are far apart but the project is slowly progressing. This update is about two of the main boards of the transceiver. I decided to make this transceiver a modular design where functions are separated into modules that can be tested and verified separately. This makes it possible for me to experiment with different designs and improve the construction piece by piece. Lets bring back the original block diagram of the transceiver. The shadowed areas are the ones I'm describing here. Mixer and amplifier board on the left and product detector and modulator board on the right.
Mixer and amplifier boardThis board contains the double balanced mixer and four identical amplifier stages. Two stages for RX and two stages for TX so that for both RX and TX there are one amplifier of each side of the mixer. Which of the stages than are activated is controlled by their respective supply voltage. One supply for RX and one supply for TX. Also between every stage there is a diode switch, also controlled by the supply voltage. The board layout is more or less the same as the schematic so it should be relatively easy to follow. Input and output signals are connected using PCB edge SMA connectors. Top left connector is RF in/out, right is the IF in/out and bottom left is the mixer frequency input directly from the Adafruit Si5351 board. For simplicity I have left out the amplifier schematics in the schematic above. Contents of every amplifier block is seen below. It's the same as the amplifier that I designed in part one of this transceiver project. This amplifier could be substituted to some other 50 ohm amplifier such as an MMIC.
Product detector and modulator boardThis board contains two more of the previous amplifier stages. One in each direction in the left side of the board. They too are switched in and out depending on the RX/TX supply voltage. The product detector in the top right quadrant of the board is unchanged from my initial design. It's working very good and has a very high dynamic range. Left SMA connector is the IF in/out and the bottom right SMA connector is the BFO input from the Adafruit Si5351 board. The latest addition to the transceiver is on this board. It's the DSB modulator in the lower right quadrant of the board. (Now my transceiver can finally transmit!) The DSB signal is passed through the crystal filter (between this board and the mixer board) to filter out either the upper or lover sideband depending on the placement of the BFO frequency in regards to the filter pass band. I decided to use an NE602 for the DSB modulator. It's a very common IC for this purpose. The MC1496 may be better performing but I wanted to give the NE602 a try. I mentioned earlier that I did not want to use the NE602 in the product detector because of it's limited dynamic range. That limitation still apply here but in this case it's okay, I want the SSB transmission to be processed with controlled dynamics anyway to give high output a readability on the receiving end. My use of the NE602 is however a bit different from most designs I've seen. The output side is simple. It's just a matching transformer to match the NE602 output with the 50 ohm input of the amplifier. The input is a bit unusual and requires some explanation. I've been disappointed with the NE602 as a DSB modulator in the past because of it's relatively bad carrier suppression. This design tries to mitigate this a bit. One big reason for carrier leakage is bad biasing within the NE602. When used as an RF mixing stage this is not very critical but when used as a DSB modulator this is a big problem. My idea to mitigate this is to add external biasing adjustment and feed the NE602 with balanced audio to keep everything as symmetric as possible. It is also VERY important not to drive the OSC input to hard (I use a 470k series resistor in this design). VR1 sets the bias balance by adjusting the DC levels of IN_A and IN_B of the NE602. DC levels should be equal on both inputs for minimum carrier output. Starting off by centering VR1 and then slowly adjusting to find the spot where the carrier is as low as possible. A multi-turn potentiometer really helps here. Two TL072 op-amps creates the balanced signal for the NE602. The modulation input can take a condenser microphone directly (just need a bias resistor) for a complete modulator. I need to add some kind of speech processor also but that's a future update. The inside is a bit bit messy right now but it works! Controller board has been re-designed and the headphone amplifier is just temporary. I will add another analog board to contain the audio processing and filtration along with a proper headphone/speaker amplifier. Next up is filtering. I need some suitable band pass filtering and RF power amplifier. Right now using just the IF amplifiers the output power peaks at around 1mW. Not much but enough to be picked up on a nearby receiver. One thing I also added was a regulated power supply with RX TX switching. I decided to use linear regulators since they are generally quieter and doesn't generate spurs. They are ineffective and will require cooling but to me it's an okay trade-off.
2017-11-11 12:39 UTC
For those of you who, just as me, are bothered with the bad composite video output of the Amiga 600/1200 I may have found a very effective upgrade. For some time I have wondered why the output lately seemed to be getting significantly worse as years went by. At first I thought it was just never monitors that had bad analog video circuitry compared to older ones but recently I definitely noticed a significant degrading in video output on my machines. The big plague that is tormenting all of these machines are the leaking electrolytic capacitors. They have come to an age now where generally all of hem are affected and must be dealt with as soon as possible to no be destroyed from within! The replacement of capacitors are an absolute necessity but it's something I won't cover in this article. There are plenty of existing guides out there already. At first a thought that this whole capacitor situation was to blame for the picture degradation but after re-capping two PAL Amiga 600 machines without any obvious improvement in the video output, in fact the re-capping actually made things worse, I began to wonder what was really going on. Both machines eventually started having problem with the reg/green color phase and color flickering. A common problem when the color delay is acting up. Remember, PAL inverts the color phase on every other line and if that phase is unstable it will result in color flickering. This red/green flickering situation got me wondering about how the video encoder chip in the Amiga 600 actually worked. It uses the CXA1145 chip by Sony. The same chip is used in many other consumer equipment and video game consoles from around that time. The CXA1145 design relies on two specifically delicate external components. One delay line and one band pass filter. Those were my primary suspects. In the Amiga 600 design they are called Z221 and Z222. The datasheet is pretty vague regarding these components. It mentions that the BPF should be centered around 4.43MHz for PAL and 3.58 MHz for NTSC, the color carrier frequencies, and the delay line should be 180ns in both cases. This is a picture over the area covering the video encoder circuit of an Amiga 600. Commodore did not use the filter and delay line mentioned in the CXA1145 datasheet but instead some other ones (the rectangular shaped RF cans on the left side of the picture). Actually the delay line consists of three cans connected together in serial and the band pass filter is a single can. I have not managed to properly identify them nor source new ones and they are pretty tricky to de-solder so I would rather avoid that. On both the Amiga 600 I recently restored I had to clean the area around Z221 and Z222 thoroughly to get rid of the leaked-out and corrosive capacitor electrolyte. In the picture above the area has been cleaned and all electrolytic capacitors replaced. I suspect my thorough cleaning is part of what quickly worsened my video problems. My theory is that Z221 and Z222 cans are sensitive to various cleaning detergents. In my case I think my use of isopropyl alcohol in the area is to blame. I could clearly see the picture degrading after cleaning. The components Commodore used for Z221 and Z222 filter and delay line may not be aging gracefully and are not robust enough to survive a leaky capacitor and cleanup. In fact I suspect the leaking capacitor electrolyte itself may even deteriorate the Z221 and Z222. They have a typical RF can style construction with an open bottom against the PCB. Any fluids on the PCB will effectively move up into the can because of the capillary force. This got me thinking about a possible replacement for the Amiga 600/1200 analog video section. Preferably a solution that would fit inside the case and one that also had to be fully reversible. This turned out to be easier than I initially thought! The diagram above is taken from the CXA1145 datasheet and shows the example circuit suggested by Sony and is basically the same design used by Commodore in the Amiga 600/1200. The bandpass filter and delay line are marked with red. Sony made an updated video encoder chip named CXA1645 that is almost pin-compatible with the CXA1145 and it does not depend on external bandpass filter and delay line! Instead those pins are used to provide a real S-video output. This is the suggested CXA1645 circuit from the datasheet. Very similar to the CXA1145. I have color coded the main differences. Red - pins with new function compared to existing circuit Blue - new or altered components Green - S-video output The changes required compared to the original circuit is pretty small. What I did was to remove the old CXA1145, replace it with a new CXA1645 and piggy-back the new and changed components around it. Worth mentioning in this case is that the CXA1145 has an audio buffer between pins 8 and 9. Those pins have other functions on the CXA1645, thus the audio to the RF modulator is lost by doing this mod. To me that was no issue but if you really want to keep the RF modulator audio working you could probably just bridge pins 8 and 9 together on the back of the PCB in order to restore that function. DISCLAIMER: This is NOT a beginners modification. It requires proper equipment and good surface mounted soldering skills. Make sure you know what you are doing and if in any doubt, leave it to someone who have the proper skills and tools. These computers are not made any more and are only getting fewer and fewer. Do not take any chances. This is only a documentation of what I did. I can not be held responsible for any eventual damages you may cause by performing this modification yourself. Step one - Removal of the CXA1145. Step two - Isolate changed pins with kapton tape. Placement of tape marked in red. Step three - Solder the new CXA1645 on top. Step four - Add components. The 1% 16k resistor between pin 18 and ground sets the color signal amplitude. I only had 5% resistors so I used one 15k and one 1k i series and measured a few samples using a calibrated LCR mater until I got well within the 1% mark. Step five - Annoy cat with fine picture and colorful mouse pointer. This has been a very successful modification. The picture is really good now. Better than I can remember it ever was and this is only composite video. Next step is to explore the possibilities with the CXA1645 S-video output. This modification was performed on an Amiga 600 but should be more or less the same for Amiga 1200 as well.
2017-10-10 20:10 UTC
To me, a big milestone when designing a computer is that very moment when the computer is capable of editing and compiling its very own source code. One might ague that a BASIC interpreter is a way to do this but I wanted to take this even further. This is a big event in my MC3 development. After completing the first really usable version of my file system MCFS2 I began working on two crucial accompanying applications, a line editor and an assembler. These two have now reached a level where they are actually usable.[show older articles]
Line editorBefore there were video terminals the only real teminal-like way of interacting with a computer was through a paper teleprinter, commonly known as a "teletype". Without a video terminal there is no way to move a cursor around. You can only enter commands one line at a time and the result will be printed line by line on a roll of paper. A line editor is a way to edit a text file using this environment. It's a bit like writing a BASIC program actually. Every line of the text file has a number and lines are altered by replacing the entire line. I'm not using a teleprinter but the principle is quite easy to implement and the neat thing is that it will work regardless of the terminal used since no special control codes are needed (unlike a modern editor like emacs or vim). My line editor is very simple. The editor can be started without arguments to create an empty file or with a file name argument to load a file into memory for editing. The editor have two modes. A command mode and a text entry mode. When started it enters the command mode indicated by a "::" prompt. Typing "help" and <enter> shows the available commands. Commands are not case sensitive and can be shortened by typing only its first letter. For example the command "s" is the same as "save". At any point, pressing ESC ($1b) will put the editor back in command mode ("::" prompt).
/devel/ # edit myfile.asm :: help Insert <line number> Delete <line number> Print <starting line number> Save Help Numbers (on/off) Quit ::There are currently two ways of manipulating text. By inserting a line and by deleting a line. Below is an example of entering a few lines of text. For clarity, I have also typed in the line-ending key strokes.
:: insert 0000 : This is a line of text <enter> 0001 : This is another line of text <enter> 0002 : Third line of text <ESC> ::The insert command can also be followed by a line number to insert a line number to insert lines at a specific place in the file.
:: insert 2 0002 : Text squeezed in between existing lines <enter> 0003 : Another line <ESC> ::The print command will print the current file to screen. It's somewhat like the "list" command in BASIC.
:: print 0000 : This is a line of text 0001 : This is another line of text 0002 : Text squeezed in between existing lines 0003 : Another line 0004 : Third line of text ::It will print a page of up to 20 lines at once and pressing <enter> will print another 20 line page of text. Pressing <ESC> will abort printout and return to command input. The print command can be followed by a line number to begin printing at a specific point of the file. The save command will prompt the user for a filename and save he file to the current directory. If the editor was used edit an existing file, the suggested filename will be the same as the opened file. Saving will always overwrites existing files so some care must be taken when naming files. The number command will turn line number display on and off. Also, line numbers are entered and displayed in HEX format. I may change that in the future. Download the MC3 line editor: - source - listing - s19
AssemblerThis really is a core piece of software in the MC3 project. I tried to make the code simple to follow rather than effective so it will not win any prizes for speed but it works really well. I wanted to make an assembler that I could use instead of the cross assembler on my PC. It had to be useful and reliable and I think I managed to achieve that. The assembler is made for 6301/6303 CPUs but can be used for the 6800 as well if only 6800 opcodes are used. This is the quick help that the assembler prints out if no arguments are given.
/devel/ # asm 6301/6303 assembler v1.0 Daniel Tufvesson 2017 Syntax: asm [files] [-l] [-e] [-d] [-tXXXX] Options: -l Generate listing -e Hide errors -d Assemble directly to memory -tXXXX Move symbol table to address $XXXX (default $0100) /devel/ #The basic principle of operation of the assembler is that it sequentially assembles the files given on the command line in two passes. Only a small part of the source is kept in memory at once to make room for as much generated program code as possible. This results in a lot of small file system reads and slows down the process substantially but enables assembly of large source files that would not normally fit in memory, I think it's a reasonable compromise. The first pass identifies all labels and builds up the symbol table in memory. The second pass completes the final assembly. Source files must be defined before any options. Example:
/devel/ # asm file1.asm file2.asm file3.asm -l -d -t4000Without any options the assembler only builds up the symbol table and verifies the assembly. No program listing or machine code is generated. Good for verifying source code without using up unnecessary memory. Errors are reported and printed to the screen. To actually generate anything, an option must be given. See explanation of options below. Generate listing (option -l) This option prints out the program listing. It will show the filename, line number, assembly line and generated data. When the assembly is complete it will end with printing the symbol table. Assemble directly to memory (option -d) This option will tell the assembler to write the generated machine code and data directly to memory. Move symbol table (option -tXXXX) When assembling code to memory there is a risk that the symbol table will collide with the generated data. Therefore this option allows for moving the start of the symbol table to another place in memory rather than the default $0100. This is the only option that takes an argument. A four character 16 bit hexadecimal address must follow directly after the "-t". Hide errors (option -e) This option will hide all errors from the printout. They will however still show up in the error counter. As you see from the options, the assembler generates machine code directly to memory. Right now it was the easiest and yet most flexible method to implement. It leaves the program directly executable in memory directly after assembly for fast turn-around time when developing. In the future I will probably extend the functionality to generate S19 files directly. Right now the easiest way to save an executable program back to the file system is to assemble to memory and then save the generated memory range to file. This method also memory efficient since S19 data takes up a lot of space.