Home Why Does ‘member’ Behave Differently in FiveAM?

Uncategorized

Why Does ‘member’ Behave Differently in FiveAM?

Learn why the Common Lisp ‘member’ function behaves unexpectedly in FiveAM tests due to string coalescing in SBCL environments.

byDev Solutions

January 11, 2026

Split image showing Common Lisp 'member' function passing in runtime but failing in FiveAM test with dramatic comparison and bright code highlights

🧠 Problems with object identity in SBCL can make FiveAM tests fail when you don't expect it.
⚠️ By default, member uses eql. This means it checks if two strings are the exact same object in memory, not if their text content is the same.
🔍 FiveAM compiles test files on their own. This changes how identical strings are stored in memory.
✅ Use :test #'equal or :test #'string= to prevent tricky bugs when comparing strings.
🛠️ Helper functions make tests more reliable and stop wrong ideas about equality.

Why Good Functions Go Wrong: What Equality Means for Testing in Common Lisp

You are making tests in FiveAM for your Common Lisp project. Everything seems fine. But then, a basic test comparing strings with member suddenly fails for a reason you don't see right away. Your REPL works fine with (member "hello" '("hello")). But the same code inside a FiveAM test gives NIL. If this confuses you, many people feel the same way. This article tells you why this happens in SBCL (Steel Bank Common Lisp). And it explains how string identity and equality checks play a part in this problem. Then, it shows how to write stronger tests using FiveAM.

Understanding `member` in Common Lisp

In Common Lisp, member is a general function for searching lists. People often use it to check if something is in a list. It's strong, but also tricky, because it uses different ways to check for a match.

Basic Usage

(member "hello" '("hello" "world"))

This returns ("hello" "world"). This happens if you are in the REPL and using SBCL. But really, this works because of how the interpreter sees equality.

By default, member uses eql as its test rule. For many kinds of data, this does what you think it will:

Numbers: (eql 5 5) returns T
Characters: (eql #\A #\A) returns T

But with strings and other things made of other parts (like lists or structures), eql sees if they are the exact same object, not if their content is the same:

(eql "hello" "hello") ; May return NIL depending on string allocation
(equal "hello" "hello") ; Always returns T

This difference is very important when testing strings. If the two strings "hello" are in different spots in memory, eql will see them as not equal. This is true even if their text is the same.

Specifying Equality Functions

You can change how it usually works by giving it a different test function to use:

(member "hello" '("hello" "world") :test #'equal)

Common ways to check for equality include:

eq: This is the most strict. It is true only if the two things are the very same object.
eql: This is a little less strict. It also finds numbers and characters that have the same value.
equal: This looks at structure and goes deeper. It compares the content of strings, lists, and other things.
equalp: This is even less strict. It does not care about letter case for strings and ignores the type of numbers.

Observable Differences in FiveAM

Your REPL says everything works. But then FiveAM tells you something else:

;; REPL
(member "test" '("test")) ; => ("test")

;; FiveAM test
(is (member "test" '("test"))) ; Assertion fails

What you see is a sign of how compiling code and how memory is used at runtime are different. This is very important when you use SBCL and FiveAM together. The problem is not with your code's thinking, but with how objects are identified.

SBCL’s Role: String Interning and Memory Allocation

SBCL is a very popular and fast Common Lisp compiler. It does many smart things when it compiles code. Some of these change how strings are handled.

String Interning in SBCL

"Interning" means using one main copy of a string literal again and again. SBCL might do this:

Reuse a string object if possible during compilation.
Create a new string object if the same text shows up in different parts of the code being compiled.

In simple terms:

(let ((a "test")
      (b "test"))
  (eql a b)) ; May return NIL

Even if a and b look the same, they might be separate objects in memory. When SBCL compiles source files on their own (like it does with FiveAM), it might make new objects for each string literal.

From the SBCL Manual:

"Programs might combine certain objects that don't change, like strings or numbers. But this is not promised for strings."

The FiveAM Testing Environment

FiveAM compiles and loads test files on their own, not as part of the main program code. According to the FiveAM GitHub Wiki:

“Test files are compiled and loaded on their own. This can affect how shared objects like strings are identified if you are not careful.”

Each compiled test file might make its own copy of a literal string. So, if a test uses eql to compare "test" from the test suite to "test" defined in the main program, the comparison will fail if they are in different spots in memory.

Example: When Equality Functions Collide

Let's look at a real example:

(defun test-member-eql ()
  (let ((a "test")
        (b "test"))
    (print (eql a b))        ; Likely NIL
    (print (equal a b))      ; T
    (print (member a (list b))) ; NIL
))

(test-member-eql)

This shows three things:

eql fails because it looks for the exact same object.
equal works because it looks at the text content.
member uses eql by default, so it returns NIL.

Now, change the member call:

(member a (list b) :test #'equal) ; Returns ("test")

Or, for comparing strings, even better:

(member a (list b) :test #'string=) ; Also returns ("test")

These changes make the test reliable and can work anywhere.

Safe Usage Guidelines for `member` in Tests

To avoid hard-to-find, confusing bugs in your tests, follow these rules when using member with strings:

✅ Do:

Always use :test #'equal or :test #'string= when you compare strings.
Know if your test rule checks content or if it's the same object.
Be clear in tests, even if the REPL seems to work fine.

❌ Don't:

Don't count on member's default behavior in any test that uses strings or lists inside other lists.
Don't assume that two pieces of text that look the same are the same object.
Don't mix string comparisons without clearly setting the test rule.

The right way to do it in FiveAM:

(is (member "foo" '("foo" "bar") :test #'equal))

The wrong and easily broken way:

(is (member "foo" '("foo" "bar")))

Discrepancies Between Compiler and Runtime

Compiler changes can change how your program works when it runs. In Common Lisp, the way code is compiled and the way it is run line-by-line can be different. This can cause tests to fail when you don't expect it.

When This Matters:

Separate Compiled Parts: FiveAM compiles test files on their own. SBCL might not share strings between them.
Code Placed Directly: If you use declaim or other compiler commands, SBCL might put code directly into other code in a different way for live systems compared to testing.
CI/CD Pipelines: These systems often test compiled code. This is different from the REPL you use every day that runs code line-by-line.

Knowing about these differences helps you have fewer tests that fail sometimes and fewer CI errors later.

Improving Test Reliability with Helper Functions

Put your comparison code in one place. This will stop you from making the same mistakes and make the code easier to read.

Define a Helper Function

(defun string-in-list-p (needle haystack)
  (member needle haystack :test #'string=))

Then, in your tests:

(is (string-in-list-p "foo" '("foo" "bar")))

Or Use a Macro for Assertions

(defmacro assert-string-member (needle haystack)
  `(is (member ,needle ,haystack :test #'string=)))

(assert-string-member "foo" '("foo" "bar"))

This way of doing things:

Prevents careless mistakes.
Helps you use the same code again.
Makes things clearer, especially for teams new to how Lisp checks for equality.

Best Practices for Lisp Testing

Here are some basic ways to test more of your code and stop errors that you don't see:

Always say which :test to use for functions like member, assoc, find, and others.
Run test code yourself in the REPL to see if they match what you think.
Save values and check their identity clearly with eq, eql, and equal.
Use format or print to see what's happening when you debug.
Don't write string text right into code that checks if objects are the same.

Creating Bulletproof Common Lisp Tests

Common Lisp is strong, but its flexibility can cause problems. What looks right in the REPL might work differently when compiled or tested. Finding problems like these, especially those based on small differences between eql and equal, means knowing how equality functions work with the compiler.

Clearly state your equality rules. Make helper functions to make comparisons standard. Understand how tools like SBCL and FiveAM work. When you do these things, your tests will be correct and steady across different systems and setups.

Broader Implications for Lisp Teams

As Lisp projects get bigger and CI systems become a part of daily work, these small bugs can make people trust automated testing less. Tests must be right and predictable. Tests that fail because of how eql versus equal work might pass or fail depending on compiler settings or how a string is created.

Engineering teams should:

Use style rules that say you must use clear equality checks.
Have test setups that are as close to the real system as possible.
Use integration tests with unit tests to find tricky compiler or runtime problems.

This simple lesson in string comparison shows more about the clever and risky power of Common Lisp. Being aware, working carefully, and using the right tools make all the difference.