Hi all,

Apologies for the long overdue update, I’ve emerged from the lab after **checks notes*…* 2 months?? I finally fought some of these issues to a standstill and got some time to sit down and write about it in a more legible form than the increasingly exasperated lab-journals that are the github issues.

Rev 5 Hand Build and Smoke Test

I won’t get all Christopher Nolan on you here, so we’ll keep things mostly chronological and start with how I built these four units and brought them up. I had made a test plan beforehand to stay organized while bringing these up for the first time.

The build took a few days under the microscope with tweezers for top and bottom SMT assembly, plus a day of through hole soldering and cleaning (ultrasonic cleaner really put in work here). I did notice one mistake right away:

SC-70, not SOT-23, classic. Continuing on my documentation kick, I filed this under issue #304, ordered the proper replacement, and forged on. Here’s one of the boards fully assembled:

I measured the impedances on all the rails and to my relief there was no smoke when it came time to power these up… Until the main buck regulator for the PCIe unit (unit 4) stopped working!

I thought it might have been ESD, but Andrew caught that I was exceeding the max voltage on the EN pin - D’oh! Easy fix though, I just bodged in a divider and never smoked this regulator again:

As opposed to smoking a regulator, I ran into an issue where the regulator could never be turned off! Thankfully fixing this was as easy is bodging a resistor in parallel with a capacitor:

They’re so cosy! That was it for the bring up, so I started building these units into enclosures for functional testing.

Assembly Instructions for Humans: You Thought CISC was Bad?

“Wait! I should probably document how to build one of these things” After all, I not only want folks to be able to build these themselves, but I also need instructions to show my friends once I inevitably draft them for box build.

So I figured out how to get a documentation website up (and by “figured out”, I mean “copied Glasgow’s homework”). I have a very comprehensive guide to building a TS-USB4 unit (Thunderbolt/USB4 Variant) up on the docs website and I highly recommend checking it out - it’s got loads of great photos and showcases a lot of the work that went into making this a robust product that’s as easy as possible to assemble. Here’s what a finished unit looks like:

Functional Testing

With the assembly process documented, it was time to see if these units worked as oscilloscopes. In the functional test issue, we ran into some problems with the gateware, which was quickly fixed by Nate, who helped me debug the issue with me - Thanks Nate! I also made a mistake with the labels on the schematic, but thankfully with an FPGA that was no problem to fix. It was at this point, I made the mistake of “quickly testing” the noise performance out of curiosity. But first, at some point during my week-long descent into madness, I received a package… In a big truck… on pallets… Which was a really cool experience!

A Flock of Pelicans Arrive at EEVengers HQ

After much back and forth with two different vendors (one for the foam, one for the case and assembly), the 450 custom Pelican cases came in - and they’re soo nice. Best part is I can use all the boxes when it comes time to ship them to Mouser, just have to slap a label over the existing label and that’s it.

Chasing Microvolts

Ever seen a github issue with >200 comments? Ever seen a github issue with >200 comments by the same person? Ever seen an electrical engineer slowly lose their mind!? Check it out, but I’ll spare you a long rant and give you the gist here.

The first big realization is the PCIe unit needs more shielding and a better main buck regulator, located further away from the front end. A higher switching frequency buck, though counter-intuitive, will allow for the rail to be filtered much more effectively. Placing the buck on the bottom side of the board will also reduce the noise that is coupled into the front end circuits on the top side of the board.

One fun thing I tried was using the interposer from the USB4 variant to shield the bottom side of the PCIe variant:

That didn’t help much, I suspect the noise is getting on on the top side, by the BNCs. I have a similar idea to use a PCB as a shield there that I’ll be testing later. To eliminate the shielding issue, I decided to conduct the rest of the tests on the USB4 variant - which is very well shielded!

There are two things that actually improved the noise on this unit, everything else was a red herring. Any perceived improvement could be chalked up to how hot the unit was / how long it was running before the test.

The first thing that worked was simply filtering the output of the charge pump with a ferrite bead and a few big chonky caps. This killed the peak in the FFT associated with the 2MHz switching frequency of the charge pump.

The second thing that worked… was much weirder. There was too much capacitive load on the opamp, which would cause overshoot if it was buffering a signal, but in our case it was just buffering a fixed voltage.

Turns out, it exhibits very very tiny oscillations at 653.5 kHz, not enough to be seen on a scope when probing -VBIAS directly, but enough to be picked up on the FFT once it’s amplified 87x by the front end circuit on the most sensitive voltage range.

Adding a 100 Ohm series resistor made that peak disappear, leaving us with a delightfully low noise floor in the range of most switching regulators.

This means that any peak you see when using ThunderScope to measure your power supply noise is actually part of your circuit!

There is some bad news though. I had finally got the RMS noise to measure 50uV as described in a previous update and then realized the gain calibration factor was much worse than I assumed. There was also the fact that the noise got worse with temperature! Another fun issue I ran into was ngscopeclient not calculating ACRMS correctly for longer captures. I found this out when the noise decreased when capture length increased, when it should converge to a slightly higher value. Andrew recognized it as a known issue with floating point addition on large datasets as soon as he caught wind of it and fixed it by applying Kahan’s algorithm.

All together, it looks like our noise performance is actually somewhere around 70uVrms at full 500 MHz bandwidth and most sensitive range. This is still very good and is neck and neck with the excellent Siglent HD series. It also beats our stated spec of 80uVrms, but I’d be lying if I said it didn’t sting knowing that we weren’t beating any records this time around. I have lots of ideas for next time though, but for now I’ll take my lumps and move on.

The Quest for Signed Drivers Continues

After jumping through a few hoops, I have an EV code signing certificate! All I need to do is log into my Microsoft Hardware Partner account:

Ughhh. Now I need to defeat the support chatbot in hand to hand combat and then make a [Speech 100] check with some customer support tech. Oh well, at least this should be the last step for the Windows drivers. Still need to look into which CNC machined aluminium hoops I need to jump through to get the Apple drivers signed.

The Attenuator Trolley Problem

I’ve mentioned this attenuator in a previous update. The response was a bit wobbly, but it proved that a two stage attenuator design could meet our 500 MHz bandwidth target. I went to test the frequency response (caution: hundreds of graphs) and after a little bit of fiddling on Rev 5 I got close to what we saw before:

Alright, the bandwidth is a bit low, and the peaking after the dip is bad - that’s still something I knew roughly how to fix. However, I noticed something weird while adjusting the trimcap and compensating the attenuator (same theory as probe compensation):

That is a very unsquare wave. I tried the single stage attenuator from Rev 4.1 and it didn’t have this issue (taking the output straight from the front end into a benchtop scope here):

Just go back to the old attenuator then, easy! Not so fast, there’s a reason I named this section after the trolley problem. The bandwidth of this circuit is dreadful, which is the problem this new attenuator was supposed to solve:

What if we did a single stage attenuator but used much less capacitance?

We can see the issue creeping up on us here in the bode plot, but overall bandwidth is still far too low at 256MHz:

At least we can make a working theory about what’s going on now. The higher the impedance of the lower leg of the attenuator, the worse this low frequency dip is. This impedance is the 20kOhm lower leg resistor in parallel with the lower leg capacitor. The capacitor would need to be larger to get this lower impedance, but the larger the cap is, the worse the bandwidth. So we have our hostages on the tracks, but what exactly is the train in this situation? There must be something causing this weird interaction with the lower leg impedance!

The Relays? Again??

At some point in my roaring rampage of rework, I damaged the relays with all the hot air going around and had to remove them. Out of curiosity I just shorted the relay terminals together and tested another attenuator circuit. It didn’t have this issue!

I was, of course, immediately suspicious. I added the attenuator relay back in and put a dent in the low frequency response once more:

To really narrow it down, I lifted the side of the relay on the lower leg of the attenuator and shorted the terminals again:

Lo and behold, the issue went away:

We have our culprit, so it was time to try some new relays!

Attenuator Design: Episode IV - A New Hope

Nope. None of the five viable alternate relays tested improved the low frequency issue. The RF one I tested out of curiosity improved it slightly, but it is way too expensive to use on this design. There is some unexpected, but good, news though. One of the relays (KEMET UD2-4.5NUN-L) raised the bandwidth of the attenuator from 442 MHz to 682 MHz! This gives us some slack to play around with.

Capacitive Confusion

At this point, I chose to pause and collect the facts together to take a more organized approach to optimizing the front end circuit. This github issue has a great summary that I’m not going to get into here.

Having solved the bandwidth issue with the new relay, I chose to test the capacitance of the input. I was aiming to make the input capacitance in the range of 12-15pF. The reason for this is to keep compatibility with most 10x passive scope probes. Low end probes will not compensate for a 10pF input, while higher end probes will not compensate for a 18pF input. It is very important for both 1x and 50x attenuation paths to have the same capacitance, so that the user doesn’t need to re-compensate their probes when switching voltage ranges.

The way I had been testing this is by assuming the front end acted like a 1st order low pass filter and back-calculating the capacitance from the -3dB cutoff and the series resistance of the input. This worked in theory, but failed in practice! I found this out when moving the probe (which was compensated for the front end circuit) from the front end circuit to the SDS804X scope, which has 17pF of input capacitance - and it kept its compensation fairly well? This seems to point to the capacitance being around 17pF, not the 12pF that I thought.

I decided to measure the capacitance some other way, using these LCR tweezers I had. This proved incredibly frustrating so I tore them apart and made a scope design specific LCR meter:

To get the capacitance I wanted out of the circuit, I had to remove the additional capacitor I added to set the input capacitance. Turns out the circuit already had about the right capacitance without intentionally adding more!

Probe Problems

This led to another issue, the response with a probe connected was much peakier with the attenuator on than with it off:

Adding a series resistor to the input of the attenuator makes the response less peaky:

This trades off bandwidth on 50 Ohm mode however, which adds another trade-off to the 6 dimensional attenuator design trade-off graph. I was pretty satisfied now that I knew exactly which variables to change to really dial this in. It was at this point that I was considering making a test board, or a whole Rev 5.1. This next issue really sealed that deal…

Coupling Consistently

At some point during testing, I noticed that the low frequency response differed between AC and DC coupling. This got much worse with a probe:

This comment has a good explanation of the issue in the time domain. The solution was to vary this one capacitor in the feedback path of the low-frequency path opamp for each coupling mode. I tested this with an analog switch dead bugged on top of the opamp:

Once I nailed down the capacitor values and tweaked a few more things, the response was pretty good and the change in response between AC and DC coupling was minimal!

A change like this is officially big enough to warrant a revision. So what do we do now?

Next Steps

This basically sets us back to where we were last update.

1. Roll in Changes from Testing: This should take about a week, there aren’t that many changes but I will have to rip up some of the front end to fit the new analog switch.

2. Start Working on Test Fixtures: Should be able to get a quick turn board made to get a rough bed of nails prototype done by the time the Rev 5.1 boards come in.

3. Hand Build Rev. 5.1: Boards will take 2-3 weeks to be made, and a few days to assemble.

4. Test Rev. 5.1: Initial bring-up will be even smoother than Rev 5, which lets me get straight into validating the improvement in noise performance and doing the last tweaks on the front end.

Once these are done, I can roll in the component value changes into a Rev 5.2. I doubt there will be any copper changes at this point, especially anything major enough to do another hand build.

Next up,

1. Get Dev. Edition PCBs Assembled: send the design to a CM and wait a month or so.

2. Finish Test Fixtures and Assembly Jigs: use the time the CM is working on the PCBAs.

3. Test and Ship Dev. Edition units: these should reach backers in late September.

Thanks for all of your patience, I hope this update has been a good look into how hard it is to make a reliable scope front end, I won’t ship anything less than that and I hope it’s worth the wait. Can’t wait to share some good news with you soon!

Cheers,

Aleksa