Merge ~chrispitude/beautifulsoup:more-modular-soupstrainers-doc into beautifulsoup:more-modular-soupstrainers

[Chris Papademetrious]

add an example for the new ElementFilter feature

Unmerged commits

b17956d... by Chris Papademetrious on 2024-02-03

add documentation example for ElementFilter

Signed-off-by: Chris Papademetrious <email address hidden>

4e37196... by Leonard Richardson on 2024-02-02

Added some basic typing to test methods that are called by the actual tests.

fe88554... by Leonard Richardson on 2024-02-02

Clarified the match protocol thing.

2618f24... by Leonard Richardson on 2024-02-02

Decided putting limit in ResultSet wasn't a good idea.

cc290e9... by Leonard Richardson on 2024-02-02

Added note that Protocol is in typing_extensions so it could be used now.

4c0618d... by Leonard Richardson on 2024-02-02

Made some typing system changes necessary to get the code to run under Python 3.8.

d5db19b... by Leonard Richardson on 2024-02-02

I feel like it's more consistent to use modified_attrs throughout.

792192c... by Leonard Richardson on 2024-02-02

Sorted out the _RawAttributeValue/_AttributeValue typing issue.

814f1a6... by Leonard Richardson on 2024-02-02

All the easy stuff is resolved now, and most of the medium-sized stuff except an issue with _RawAttributeValues versus _AttributeValues.

3898972... by Leonard Richardson on 2024-02-01

Got the type definitions in _deprecation.py a _little_ better.

diff --git a/CHANGELOG b/CHANGELOG
index 162e3dc..41b1467 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,7 +1,5 @@
 = 4.13.0 (Unreleased)
 
-TODO: we could stand to put limit inside ResultSet
-
 * This version drops support for Python 3.6. The minimum supported
   major Python version for Beautiful Soup is now Python 3.7.
 
@@ -33,12 +31,15 @@ TODO: we could stand to put limit inside ResultSet
   you, since you probably use HTMLParserTreeBuilder, not
   BeautifulSoupHTMLParser directly.
 
-* The TreeBuilderForHtml5lib methods fragmentClass and getFragment
-  now raise NotImplementedError. These methods are called only by
-  html5lib's HTMLParser.parseFragment() method, which Beautiful Soup
-  doesn't use, so they were untested and should have never been called.
-  The getFragment() implementation was also slightly incorrect in a way
-  that should have caused obvious problems for anyone using it.
+* The TreeBuilderForHtml5lib methods fragmentClass(), getFragment(),
+  and testSerializer() now raise NotImplementedError. These methods
+  are called only by html5lib's test suite, and Beautiful Soup isn't
+  integrated into that test suite, so this code was long since unused and
+  untested.
+
+  These methods are _not_ deprecated, since they are methods defined by
+  html5lib. They may one day have real implementations, as part of a future
+  effort to integrate Beautiful Soup into html5lib's test suite.
 
 * If Tag.get_attribute_list() is used to access an attribute that's not set,
   the return value is now an empty list rather than [None].
@@ -73,6 +74,11 @@ TODO: we could stand to put limit inside ResultSet
 * A SoupStrainer can now filter tag creation based on a tag's
   namespaced name. Previously only the unqualified name could be used.
 
+* Some of the arguments in the methods of LXMLTreeBuilderForXML
+  have been renamed for consistency with the names lxml uses for those
+  arguments in the superclass. This won't affect you unless you were
+  calling methods like LXMLTreeBuilderForXML.start() directly.
+
 * All TreeBuilder constructors now take the empty_element_tags
   argument. The sets of tags found in HTMLTreeBuilder.empty_element_tags and
   HTMLTreeBuilder.block_elements are now in
diff --git a/bs4/__init__.py b/bs4/__init__.py
index 95bd48d..6f01a75 100644
--- a/bs4/__init__.py
+++ b/bs4/__init__.py
@@ -72,7 +72,7 @@ from typing import (
     cast,
     Counter as CounterType,
     Dict,
-    Iterable,
+    Iterator,
     List,
     Sequence,
     Optional,
@@ -84,10 +84,11 @@ from typing import (
 
 from bs4._typing import (
     _AttributeValue,
-    _AttributeValues,
     _Encoding,
     _Encodings,
     _IncomingMarkup,
+    _RawAttributeValue,
+    _RawAttributeValues,
     _RawMarkup,
 )
 
@@ -451,7 +452,7 @@ class BeautifulSoup(Tag):
         clone.original_encoding = self.original_encoding
         return clone
         
-    def __getstate__(self) -> dict[str, Any]:
+    def __getstate__(self) -> Dict[str, Any]:
         # Frequently a tree builder can't be pickled.
         d = dict(self.__dict__)
         if 'builder' in d and d['builder'] is not None and not self.builder.picklable:
@@ -467,7 +468,7 @@ class BeautifulSoup(Tag):
             del d['_most_recent_element']
         return d
 
-    def __setstate__(self, state: dict[str, Any]) -> None:
+    def __setstate__(self, state: Dict[str, Any]) -> None:
         # If necessary, restore the TreeBuilder by looking it up.
         self.__dict__ = state
         if isinstance(self.builder, type):
@@ -613,11 +614,11 @@ class BeautifulSoup(Tag):
             name:str,
             namespace:Optional[str]=None,
             nsprefix:Optional[str]=None,
-            attrs:_AttributeValues={},
+            attrs:_RawAttributeValues={},
             sourceline:Optional[int]=None,
             sourcepos:Optional[int]=None,
             string:Optional[str]=None,
-            **kwattrs:_AttributeValue,
+            **kwattrs:_RawAttributeValue,
     ) -> Tag:
         """Create a new Tag associated with this BeautifulSoup object.
 
@@ -664,7 +665,7 @@ class BeautifulSoup(Tag):
         # The user may want us to use some other class (hopefully a
         # custom subclass) instead of the one we'd use normally.
         container = cast(
-            type[NavigableString],
+            Type[NavigableString],
             self.element_classes.get(container, container)
         )
 
@@ -894,14 +895,16 @@ class BeautifulSoup(Tag):
 
     def handle_starttag(
             self, name:str, namespace:Optional[str],
-            nsprefix:Optional[str], attrs:_AttributeValues,
+            nsprefix:Optional[str], attrs:_RawAttributeValues,
             sourceline:Optional[int]=None, sourcepos:Optional[int]=None,
             namespaces:Optional[Dict[str, str]]=None) -> Optional[Tag]:
         """Called by the tree builder when a new tag is encountered.
 
         :param name: Name of the tag.
         :param nsprefix: Namespace prefix for the tag.
-        :param attrs: A dictionary of attribute values.
+        :param attrs: A dictionary of attribute values. Note that
+           attribute values are expected to be simple strings; processing
+           of multi-valued attributes such as "class" comes later.
         :param sourceline: The line number where this tag was found in its
             source document.
         :param sourcepos: The character position within `sourceline` where this
@@ -964,7 +967,7 @@ class BeautifulSoup(Tag):
     def decode(self, indent_level:Optional[int]=None,
                eventual_encoding:_Encoding=DEFAULT_OUTPUT_ENCODING,
                formatter:Union[Formatter,str]="minimal",
-               iterator:Optional[Iterable[PageElement]]=None,
+               iterator:Optional[Iterator[PageElement]]=None,
                **kwargs:Any) -> str:
         """Returns a string representation of the parse tree
             as a full HTML or XML document.
diff --git a/bs4/_deprecation.py b/bs4/_deprecation.py
index febc1b3..86b22a2 100644
--- a/bs4/_deprecation.py
+++ b/bs4/_deprecation.py
@@ -17,7 +17,7 @@ from typing import (
     Callable,
 )
 
-def _deprecated_alias(old_name, new_name, version):
+def _deprecated_alias(old_name:str, new_name:str, version:str):
     """Alias one attribute name to another for backward compatibility
 
     :meta private:
@@ -29,23 +29,23 @@ def _deprecated_alias(old_name, new_name, version):
         return getattr(self, new_name)
 
     @alias.setter
-    def alias(self, value:str)->Any:
+    def alias(self, value:str) -> None:
         ":meta private:"
         warnings.warn(f"Write to deprecated property {old_name}. (Replaced by {new_name}) -- Deprecated since version {version}.", DeprecationWarning, stacklevel=2)
         return setattr(self, new_name, value)
     return alias
 
-def _deprecated_function_alias(old_name:str, new_name:str, version:str) -> Callable:
-    def alias(self, *args, **kwargs):
+def _deprecated_function_alias(old_name:str, new_name:str, version:str) -> Callable[[Any], Any]:
+    def alias(self, *args:Any, **kwargs:Any) -> Any:
         ":meta private:"
         warnings.warn(f"Call to deprecated method {old_name}. (Replaced by {new_name}) -- Deprecated since version {version}.", DeprecationWarning, stacklevel=2)
         return getattr(self, new_name)(*args, **kwargs)
     return alias
 
 def _deprecated(replaced_by:str, version:str) -> Callable:
-    def deprecate(func):
+    def deprecate(func:Callable) -> Callable:
         @functools.wraps(func)
-        def with_warning(*args, **kwargs):
+        def with_warning(*args:Any, **kwargs:Any) -> Any:
             ":meta private:"
             warnings.warn(
                 f"Call to deprecated method {func.__name__}. (Replaced by {replaced_by}) -- Deprecated since version {version}.",
diff --git a/bs4/_typing.py b/bs4/_typing.py
index ab8f7a0..7fff292 100644
--- a/bs4/_typing.py
+++ b/bs4/_typing.py
@@ -8,7 +8,12 @@
 # * In 3.10, TypeAlias gains capabilities that can be used to
 #   improve the tree matching types (I don't remember what, exactly).
 # * 3.8 defines the Protocol type, which can be used to do duck typing
-#   in a statically checkable way.
+#   in a statically checkable way. (Protocols are also in typing_extensions,
+#   so I could add this now in the couple places it's needed.)
+# * In 3.9 it's possible to specialize the re.Match type,
+#   e.g. re.Match[str]. In 3.8 there's a typing.re namespace for this,
+#   but it's removed in 3.12, so to support the widest possible set of
+#   versions I'm not using it.
 
 import re
 from typing_extensions import TypeAlias
@@ -48,16 +53,22 @@ _InvertedNamespaceMapping:TypeAlias = Dict[_NamespaceURL, _NamespacePrefix]
 
 # Aliases for the attribute values associated with HTML/XML tags.
 #
-# Note that these are attribute values in their final form, as stored
-# in the `Tag` class.  Different parsers present attributes to the
-# `TreeBuilder` subclasses in different formats, which are not defined
-# here.
+# These are the relatively unprocessed values Beautiful Soup expects
+# to come from a `TreeBuilder`.
+_RawAttributeValue: TypeAlias = str
+_RawAttributeValues: TypeAlias = Dict[str, _RawAttributeValue]
+
+# These are attribute values in their final form, as stored in the
+# `Tag` class, after they have been processed and (in some cases)
+# split into lists of strings.
 _AttributeValue: TypeAlias = Union[str, Iterable[str]]
 _AttributeValues: TypeAlias = Dict[str, _AttributeValue]
 
-# The most common form in which attribute values are passed in from a
-# parser.
-_RawAttributeValues: TypeAlias = dict[str, str]
+# The methods that deal with turning _RawAttributeValues into
+# _AttributeValues may be called several times, even after the values
+# are already processed (e.g. when cloning a tag), so they need to
+# be able to acommodate both possibilities.
+_RawOrProcessedAttributeValues:TypeAlias = Union[_RawAttributeValues, _AttributeValues]
 
 # Aliases to represent the many possibilities for matching bits of a
 # parse tree.
diff --git a/bs4/builder/__init__.py b/bs4/builder/__init__.py
index b59513e..29726c3 100644
--- a/bs4/builder/__init__.py
+++ b/bs4/builder/__init__.py
@@ -33,15 +33,20 @@ from bs4.element import (
     nonwhitespace_re
 )
 
+from bs4._typing import (
+    _AttributeValues,
+    _RawAttributeValue,
+)
 if TYPE_CHECKING:
     from bs4 import BeautifulSoup
     from bs4.element import (
         NavigableString, Tag,
-        _AttributeValues, _AttributeValue,
     )
     from bs4._typing import (
+        _AttributeValue,
         _Encoding,
         _Encodings,
+        _RawOrProcessedAttributeValues,
         _RawMarkup,
     )
     
@@ -75,7 +80,7 @@ class TreeBuilderRegistry(object):
     builders_for_feature: Dict[str, List[Type[TreeBuilder]]]
     builders: List[Type[TreeBuilder]]
     
-    def __init__(self):
+    def __init__(self) -> None:
         self.builders_for_feature = defaultdict(list)
         self.builders = []
 
@@ -233,7 +238,7 @@ class TreeBuilder(object):
     #: no contents--that is, using XML rules. HTMLTreeBuilder
     #: defines a different set of DEFAULT_EMPTY_ELEMENT_TAGS based on the
     #: HTML 4 and HTML5 standards.
-    DEFAULT_EMPTY_ELEMENT_TAGS: Optional[Set] = None
+    DEFAULT_EMPTY_ELEMENT_TAGS: Optional[Set[str]] = None
     
     #: Most parsers don't keep track of line numbers.
     TRACKS_LINE_NUMBERS: bool = False
@@ -347,7 +352,7 @@ class TreeBuilder(object):
         """
         return False
 
-    def _replace_cdata_list_attribute_values(self, tag_name:str, attrs:_AttributeValues):
+    def _replace_cdata_list_attribute_values(self, tag_name:str, attrs:_RawOrProcessedAttributeValues) -> _AttributeValues:
         """When an attribute value is associated with a tag that can
         have multiple values for that attribute, convert the string
         value to a list of strings.
@@ -359,88 +364,106 @@ class TreeBuilder(object):
         :param tag_name: The name of a tag.
         :param attrs: A dictionary containing the tag's attributes.
            Any appropriate attribute values will be modified in place.
+        :return: The modified dictionary that was originally passed in.
         """
-        if not attrs:
-            return attrs
-        if self.cdata_list_attributes:
-            universal: Set[str] = self.cdata_list_attributes.get('*', set())
-            tag_specific = self.cdata_list_attributes.get(
-                tag_name.lower(), None)
-            for attr in list(attrs.keys()):
-                values: _AttributeValue
-                if attr in universal or (tag_specific and attr in tag_specific):
-                    # We have a "class"-type attribute whose string
-                    # value is a whitespace-separated list of
-                    # values. Split it into a list.
-                    value = attrs[attr]
-                    if isinstance(value, str):
-                        values = nonwhitespace_re.findall(value)
-                    else:
-                        # html5lib sometimes calls setAttributes twice
-                        # for the same tag when rearranging the parse
-                        # tree. On the second call the attribute value
-                        # here is already a list.  If this happens,
-                        # leave the value alone rather than trying to
-                        # split it again.
-                        values = value
-                    attrs[attr] = values
-        return attrs
+
+        # First, cast the attrs dict to _AttributeValues. This might
+        # not be accurate yet, but it will be by the time this method
+        # returns.
+        modified_attrs = cast(_AttributeValues, attrs)
+        if not modified_attrs or not self.cdata_list_attributes:
+            # Nothing to do.
+            return modified_attrs
+
+        # There is at least a possibility that we need to modify one of
+        # the attribute values.
+        universal: Set[str] = self.cdata_list_attributes.get('*', set())
+        tag_specific = self.cdata_list_attributes.get(
+            tag_name.lower(), None)
+        for attr in list(modified_attrs.keys()):
+            modified_value:_AttributeValue
+            if attr in universal or (tag_specific and attr in tag_specific):
+                # We have a "class"-type attribute whose string
+                # value is a whitespace-separated list of
+                # values. Split it into a list.
+                original_value:_AttributeValue = modified_attrs[attr]
+                if isinstance(original_value, _RawAttributeValue):
+                    # This is a _RawAttributeValue (a string) that
+                    # needs to be split into a list so it can be an
+                    # _AttributeValue.
+                    modified_value = nonwhitespace_re.findall(original_value)
+                else:
+                    # html5lib calls setAttributes twice for the
+                    # same tag when rearranging the parse tree. On
+                    # the second call the attribute value here is
+                    # already a list. This can also happen when a
+                    # Tag object is cloned. If this happens, leave
+                    # the value alone rather than trying to split
+                    # it again.
+                    modified_value = original_value
+                modified_attrs[attr] = modified_value
+        return modified_attrs
     
 class SAXTreeBuilder(TreeBuilder):
     """A Beautiful Soup treebuilder that listens for SAX events.
 
-    This is not currently used for anything, but it demonstrates
-    how a simple TreeBuilder would work.
+    This is not currently used for anything, and it will be removed
+    soon. It was a good idea, but it wasn't properly integrated into the
+    rest of Beautiful Soup, so there have been long stretches where it
+    hasn't worked properly.
     """
-
-    def __init__(self, *args, **kwargs):
+    def __init__(self, *args:Any, **kwargs:Any) -> None:
         warnings.warn(
-            f"The SAXTreeBuilder class was deprecated in 4.13.0. It is completely untested and probably doesn't work; use at your own risk.",
+            f"The SAXTreeBuilder class was deprecated in 4.13.0 and will be removed soon thereafter. It is completely untested and probably doesn't work; do not use it.",
                 DeprecationWarning,
                 stacklevel=2
             )
         super(SAXTreeBuilder, self).__init__(*args, **kwargs)
     
-    def feed(self, markup:_RawMarkup):
+    def feed(self, markup:_RawMarkup) -> None:
         raise NotImplementedError()
 
-    def close(self):
+    def close(self) -> None:
         pass
 
-    def startElement(self, name, attrs):
+    def startElement(self, name:str, attrs:Dict[str,str]) -> None:
         attrs = dict((key[1], value) for key, value in list(attrs.items()))
         #print("Start %s, %r" % (name, attrs))
-        self.soup.handle_starttag(name, attrs)
+        assert self.soup is not None
+        self.soup.handle_starttag(name, None, None, attrs)
 
-    def endElement(self, name):
+    def endElement(self, name:str) -> None:
         #print("End %s" % name)
+        assert self.soup is not None
         self.soup.handle_endtag(name)
 
-    def startElementNS(self, nsTuple, nodeName, attrs):
+    def startElementNS(self, nsTuple:Tuple[str,str],
+                       nodeName:str, attrs:Dict[str,str]) -> None:
         # Throw away (ns, nodeName) for now.
         self.startElement(nodeName, attrs)
 
-    def endElementNS(self, nsTuple, nodeName):
+    def endElementNS(self, nsTuple:Tuple[str,str], nodeName:str) -> None:
         # Throw away (ns, nodeName) for now.
         self.endElement(nodeName)
         #handler.endElementNS((ns, node.nodeName), node.nodeName)
 
-    def startPrefixMapping(self, prefix, nodeValue):
+    def startPrefixMapping(self, prefix:str, nodeValue:str) -> None:
         # Ignore the prefix for now.
         pass
 
-    def endPrefixMapping(self, prefix):
+    def endPrefixMapping(self, prefix:str) -> None:
         # Ignore the prefix for now.
         # handler.endPrefixMapping(prefix)
         pass
 
-    def characters(self, content):
+    def characters(self, content:str) -> None:
+        assert self.soup is not None
         self.soup.handle_data(content)
 
-    def startDocument(self):
+    def startDocument(self) -> None:
         pass
 
-    def endDocument(self):
+    def endDocument(self) -> None:
         pass
 
 
@@ -620,13 +643,13 @@ class DetectsXMLParsedAsHTML(object):
             return False
         markup = markup[:500]
         if isinstance(markup, bytes):
-            markup_b = cast(bytes, markup)
+            markup_b:bytes = markup
             looks_like_xml = (
                 markup_b.startswith(cls.XML_PREFIX_B)
                 and not cls.LOOKS_LIKE_HTML_B.search(markup)
             )
         else:
-            markup_s = cast(str, markup)
+            markup_s:str = markup
             looks_like_xml = (
                 markup_s.startswith(cls.XML_PREFIX)
                 and not cls.LOOKS_LIKE_HTML.search(markup)
@@ -650,9 +673,13 @@ class DetectsXMLParsedAsHTML(object):
         self._first_processing_instruction = None
         self._root_tag_name = None
        
-    def _document_might_be_xml(self, processing_instruction:str):
+    def _document_might_be_xml(self, processing_instruction:str) -> None:
         """Call this method when encountering an XML declaration, or a
         "processing instruction" that might be an XML declaration.
+
+        This helps Beautiful Soup detect potential issues later, if
+        the XML document turns out to be a non-XHTML document that's
+        being parsed as XML.
         """
         if (self._first_processing_instruction is not None
             or self._root_tag_name is not None):
diff --git a/bs4/builder/_html5lib.py b/bs4/builder/_html5lib.py
index 2ea556c..51e3c97 100644
--- a/bs4/builder/_html5lib.py
+++ b/bs4/builder/_html5lib.py
@@ -12,6 +12,7 @@ from typing import (
     Iterable,
     List,
     Optional,
+    TypeAlias,
     TYPE_CHECKING,
     Tuple,
     Union,
@@ -54,6 +55,7 @@ if TYPE_CHECKING:
     from bs4 import BeautifulSoup
 
 from html5lib.treebuilders import base as treebuilder_base
+from html5lib.treewalkers import base as treewalker_base
 
 
 class HTML5TreeBuilder(HTMLTreeBuilder):
@@ -138,6 +140,14 @@ class HTML5TreeBuilder(HTMLTreeBuilder):
             # HTMLBinaryInputStream.__init__.
             extra_kwargs['override_encoding'] = self.user_specified_encoding
 
+        # TODO-TYPING: typeshed stub says the second argument to
+        # HTMLParser.parse is scripting:bool, but the implementation
+        # treats scripting as one of the kwargs. scripting:bool isn't
+        # called out separately until we get down into _parse(), and
+        # there it's the fourth argument, not the second.  I'm not
+        # sure what the stub ought to look like, but I'm confident
+        # enough that it's better to leave this alone, rather than
+        # change this call to get rid of the warning.
         doc = parser.parse(markup, **extra_kwargs)
         
         # Set the character encoding detected by the tokenizer.
@@ -146,6 +156,10 @@ class HTML5TreeBuilder(HTMLTreeBuilder):
             # charEncoding to UTF-8 if it gets Unicode input.
             doc.original_encoding = None
         else:
+            # TODO-TYPING HTMLParser.tokenizer is set by
+            # HTMLParser._parse(), so it's definitely set by this
+            # point, but it's not defined as an instance variable, so
+            # this line gives a warning.
             original_encoding = parser.tokenizer.stream.charEncoding[0]
             # The encoding is an html5lib Encoding object. We want to
             # use a string for compatibility with other tree builders.
@@ -231,72 +245,35 @@ class TreeBuilderForHtml5lib(treebuilder_base.TreeBuilder):
 
     def fragmentClass(self) -> 'Element':
         """This is only used by html5lib HTMLParser.parseFragment(),
-        which is never used by Beautiful Soup."""
+        which is never used by Beautiful Soup, only by the html5lib
+        unit tests. Since we don't currently hook into those tests,
+        the implementation is left blank.
+        """
         raise NotImplementedError()
 
     def getFragment(self) -> 'Element':
-        """This is only used by html5lib HTMLParser.parseFragment,
-        which is never used by Beautiful Soup."""
+        """This is only used by the html5lib unit tests. Since we
+        don't currently hook into those tests, the implementation is
+        left blank.
+        """
         raise NotImplementedError()
 
     def appendChild(self, node:'Element') -> None:
-        # TODO: This code is not covered by the BS4 tests.
+        # TODO: This code is not covered by the BS4 tests, and
+        # apparently not triggered by the html5lib test suite either.
         self.soup.append(node.element)
 
     def getDocument(self) -> 'BeautifulSoup':
         return self.soup
 
     # TODO-TYPING: typeshed stubs are incorrect about this;
-    # cloneNode returns a str, not None.
+    # testSerializer returns a str, not None.
     def testSerializer(self, element:'Element') -> str:
-        from bs4 import BeautifulSoup
-        rv = []
-        doctype_re = re.compile(r'^(.*?)(?: PUBLIC "(.*?)"(?: "(.*?)")?| SYSTEM "(.*?)")?$')
-
-        def serializeElement(element:Union['Element', PageElement], indent=0) -> None:
-            if isinstance(element, BeautifulSoup):
-                pass
-            if isinstance(element, Doctype):
-                m = doctype_re.match(element)
-                if m is not None:
-                    name = m.group(1)
-                    if m.lastindex is not None and m.lastindex > 1:
-                        publicId = m.group(2) or ""
-                        systemId = m.group(3) or m.group(4) or ""
-                        rv.append("""|%s<!DOCTYPE %s "%s" "%s">""" %
-                                  (' ' * indent, name, publicId, systemId))
-                    else:
-                        rv.append("|%s<!DOCTYPE %s>" % (' ' * indent, name))
-                else:
-                    rv.append("|%s<!DOCTYPE >" % (' ' * indent,))
-            elif isinstance(element, Comment):
-                rv.append("|%s<!-- %s -->" % (' ' * indent, element))
-            elif isinstance(element, NavigableString):
-                rv.append("|%s\"%s\"" % (' ' * indent, element))
-            elif isinstance(element, Element):
-                if element.namespace:
-                    name = "%s %s" % (prefixes[element.namespace],
-                                      element.name)
-                else:
-                    name = element.name
-                rv.append("|%s<%s>" % (' ' * indent, name))
-                if element.attrs:
-                    attributes = []
-                    for name, value in list(element.attrs.items()):
-                        if isinstance(name, NamespacedAttribute):
-                            name = "%s %s" % (prefixes[name.namespace], name.name)
-                        if isinstance(value, list):
-                            value = " ".join(value)
-                        attributes.append((name, value))
-
-                    for name, value in sorted(attributes):
-                        rv.append('|%s%s="%s"' % (' ' * (indent + 2), name, value))
-                indent += 2
-                for child in element.children:
-                    serializeElement(child, indent)
-        serializeElement(element, 0)
-
-        return "\n".join(rv)
+        """This is only used by the html5lib unit tests. Since we
+        don't currently hook into those tests, the implementation is
+        left blank.
+        """
+        raise NotImplementedError()
 
 class AttrList(object):
     """Represents a Tag's attributes in a way compatible with html5lib."""
@@ -340,11 +317,28 @@ class AttrList(object):
     def __contains__(self, name:str) -> bool:
         return name in list(self.attrs.keys())
 
+class BeautifulSoupNode(treebuilder_base.Node):
+    element:PageElement
+    soup:'BeautifulSoup'
+    namespace:Optional[_NamespaceURL]
+
+    @property
+    def nodeType(self) -> int:
+        """Return the html5lib constant corresponding to the type of
+        the underlying DOM object.
 
-class Element(treebuilder_base.Node):
+        NOTE: This property is only accessed by the html5lib test
+        suite, not by Beautiful Soup proper.
+        """
+        raise NotImplementedError()
 
+    # TODO-TYPING: typeshed stubs are incorrect about this;
+    # cloneNode returns a new Node, not None.
+    def cloneNode(self) -> treebuilder_base.Node:
+        raise NotImplementedError()
+
+class Element(BeautifulSoupNode):
     element:Tag
-    soup:'BeautifulSoup'
     namespace:Optional[_NamespaceURL]
 
     def __init__(self, element:Tag, soup:'BeautifulSoup',
@@ -354,30 +348,20 @@ class Element(treebuilder_base.Node):
         self.soup = soup
         self.namespace = namespace
 
-    def appendChild(self, node:'Element') -> None:
-        string_child = child = None
-        if isinstance(node, str):
-            # Some other piece of code decided to pass in a string
-            # instead of creating a TextElement object to contain the
-            # string. This should not ever happen.
-            string_child = child = node
-        elif isinstance(node, Tag):
-            # Some other piece of code decided to pass in a Tag
-            # instead of creating an Element object to contain the
-            # Tag. This should not ever happen.
-            child = node
-        elif node.element.__class__ == NavigableString:
+    def appendChild(self, node:'BeautifulSoupNode') -> None:
+        string_child:Optional[NavigableString] = None
+        child:PageElement
+        if type(node.element) == NavigableString:
             string_child = child = node.element
-            node.parent = self
         else:
             child = node.element
-            node.parent = self
+        node.parent = self
 
-        if not isinstance(child, str) and child is not None and child.parent is not None:
+        if child is not None and child.parent is not None and not isinstance(child, str):
             node.element.extract()
 
         if (string_child is not None and self.element.contents
-            and self.element.contents[-1].__class__ == NavigableString):
+            and type(self.element.contents[-1]) == NavigableString):
             # We are appending a string onto another string.
             # TODO This has O(n^2) performance, for input like
             # "a</a>a</a>a</a>..."
@@ -413,18 +397,36 @@ class Element(treebuilder_base.Node):
             return {}
         return AttrList(self.element)
 
-    def setAttributes(self, attributes:Optional[Dict]) -> None:
+    # An HTML5lib attribute name may either be a single string,
+    # or a tuple (namespace, name).
+    _Html5libAttributeName: TypeAlias = Union[str, Tuple[str, str]]
+    # Now we can define the type this method accepts as a dictionary
+    # mapping those attribute names to single string values.
+    _Html5libAttributes: TypeAlias = Dict[_Html5libAttributeName, str]
+    def setAttributes(self, attributes:Optional[_Html5libAttributes]) -> None:
         if attributes is not None and len(attributes) > 0:
+
+            # Replace any namespaced attributes with
+            # NamespacedAttribute objects.
             for name, value in list(attributes.items()):
                 if isinstance(name, tuple):
                     new_name = NamespacedAttribute(*name)
                     del attributes[name]
                     attributes[new_name] = value
 
+            # We can now cast attributes to the type of Dict
+            # used by Beautiful Soup.
+            normalized_attributes = cast(_AttributeValues, attributes)
+
+            # Values for tags like 'class' came in as single strings;
+            # replace them with lists of strings as appropriate.
             self.soup.builder._replace_cdata_list_attribute_values(
-                self.name, attributes)
-            for name, value in list(attributes.items()):
-                self.element[name] = value
+                self.name, normalized_attributes)
+
+            # Then set the attributes on the Tag associated with this
+            # BeautifulSoupNode.
+            for name, value_or_values in list(normalized_attributes.items()):
+                self.element[name] = value_or_values
 
             # The attributes may contain variables that need substitution.
             # Call set_up_substitutions manually.
@@ -434,19 +436,20 @@ class Element(treebuilder_base.Node):
             self.soup.builder.set_up_substitutions(self.element)
     attributes = property(getAttributes, setAttributes)
 
-    def insertText(self, data:str, insertBefore:Optional['Element']=None) -> None:
+    def insertText(self, data:str, insertBefore:Optional['BeautifulSoupNode']=None) -> None:
         text = TextNode(self.soup.new_string(data), self.soup)
         if insertBefore:
             self.insertBefore(text, insertBefore)
         else:
             self.appendChild(text)
 
-    def insertBefore(self, node:'Element', refNode:'Element') -> None:
+    def insertBefore(self, node:'BeautifulSoupNode', refNode:'BeautifulSoupNode') -> None:
         index = self.element.index(refNode.element)
-        if (node.element.__class__ == NavigableString and self.element.contents
-            and self.element.contents[index-1].__class__ == NavigableString):
+        if (type(node.element) == NavigableString and self.element.contents
+            and type(self.element.contents[index-1]) == NavigableString):
             # (See comments in appendChild)
             old_node = self.element.contents[index-1]
+            assert type(old_node) == NavigableString
             new_str = self.soup.new_string(old_node + node.element)
             old_node.replace_with(new_str)
         else:
@@ -504,13 +507,19 @@ class Element(treebuilder_base.Node):
             # parent's last descendant. It has no .next_sibling and
             # its .next_element is whatever the previous last
             # descendant had.
-            last_childs_last_descendant = to_append[-1]._last_descendant(False, True)
+            last_childs_last_descendant = to_append[-1]._last_descendant(
+                is_initialized=False, accept_self=True
+            )
 
+            # Since we passed accept_self=True into _last_descendant,
+            # there's no possibility that the result is None.
+            assert last_childs_last_descendant is not None
             last_childs_last_descendant.next_element = new_parents_last_descendant_next_element
             if new_parents_last_descendant_next_element is not None:
-                # TODO: This code has no test coverage and I'm not sure
-                # how to get html5lib to go through this path, but it's
-                # just the other side of the previous line.
+                # TODO-COVERAGE: This code has no test coverage and
+                # I'm not sure how to get html5lib to go through this
+                # path, but it's just the other side of the previous
+                # line.
                 new_parents_last_descendant_next_element.previous_element = last_childs_last_descendant
             last_childs_last_descendant.next_sibling = None
 
@@ -526,7 +535,12 @@ class Element(treebuilder_base.Node):
         # print("FROM", self.element)
         # print("TO", new_parent_element)
 
-    # TODO: typeshed stubs are incorrect about this;
+    # TODO-TYPING: typeshed stubs are incorrect about this;
+    # hasContent returns a boolean, not None.
+    def hasContent(self) -> bool:
+        return len(self.element.contents) > 0
+
+    # TODO-TYPING: typeshed stubs are incorrect about this;
     # cloneNode returns a new Node, not None.
     def cloneNode(self) -> treebuilder_base.Node:
         tag = self.soup.new_tag(self.element.name, self.namespace)
@@ -535,24 +549,17 @@ class Element(treebuilder_base.Node):
             node.attributes[key] = value
         return node
 
-    # TODO-TYPING: typeshed stubs are incorrect about this;
-    # cloneNode returns a boolean, not None.
-    def hasContent(self) -> bool:
-        return len(self.element.contents) > 0
-
-    def getNameTuple(self) -> Tuple[str, str]:
+    def getNameTuple(self) -> Tuple[Optional[_NamespaceURL], str]:
         if self.namespace == None:
             return namespaces["html"], self.name
         else:
             return self.namespace, self.name
-
     nameTuple = property(getNameTuple)
 
-class TextNode(Element):
-    def __init__(self, element:PageElement, soup:'BeautifulSoup'):
+class TextNode(BeautifulSoupNode):
+    element:NavigableString
+
+    def __init__(self, element:NavigableString, soup:'BeautifulSoup'):
         treebuilder_base.Node.__init__(self, None)
         self.element = element
         self.soup = soup
-
-    def cloneNode(self) -> treebuilder_base.Node:
-        raise NotImplementedError()
diff --git a/bs4/builder/_htmlparser.py b/bs4/builder/_htmlparser.py
index 91cecf7..d8e21d1 100644
--- a/bs4/builder/_htmlparser.py
+++ b/bs4/builder/_htmlparser.py
@@ -49,7 +49,6 @@ if TYPE_CHECKING:
     from bs4 import BeautifulSoup
     from bs4.element import NavigableString
     from bs4._typing import (
-        _AttributeValues,
         _Encoding,
         _Encodings,
         _RawMarkup,
@@ -60,6 +59,14 @@ HTMLPARSER = 'html.parser'
 _DuplicateAttributeHandler = Callable[[Dict[str, str], str, str], None]
 
 class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
+    #: Constant to handle duplicate attributes by ignoring later values
+    #: and keeping the earlier ones.
+    REPLACE:str = 'replace'
+
+    #: Constant to handle duplicate attributes by replacing earlier values
+    #: with later ones.
+    IGNORE:str = 'ignore'
+
     """A subclass of the Python standard library's HTMLParser class, which
     listens for HTMLParser events and translates them into calls
     to Beautiful Soup's tree construction API.
@@ -73,11 +80,13 @@ class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
             the name of the duplicate attribute, and the most recent value
             encountered.
     """
-    def __init__(self, soup:BeautifulSoup, *args, **kwargs):
+    def __init__(
+            self, soup:BeautifulSoup, *args:Any,
+            on_duplicate_attribute:Union[str, _DuplicateAttributeHandler]=REPLACE,
+            **kwargs:Any
+    ):
         self.soup = soup
-        self.on_duplicate_attribute = kwargs.pop(
-            'on_duplicate_attribute', self.REPLACE
-        )
+        self.on_duplicate_attribute = on_duplicate_attribute
         HTMLParser.__init__(self, *args, **kwargs)
 
         # Keep a list of empty-element tags that were encountered
@@ -90,14 +99,6 @@ class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
         self.already_closed_empty_element = []
 
         self._initialize_xml_detector()
-    
-    #: Constant to handle duplicate attributes by replacing earlier values
-    #: with later ones.
-    IGNORE:str = 'ignore'
-
-    #: Constant to handle duplicate attributes by ignoring later values
-    #: and keeping the earlier ones.    
-    REPLACE:str = 'replace'
 
     on_duplicate_attribute:Union[str, _DuplicateAttributeHandler]
     already_closed_empty_element: List[str]
@@ -145,7 +146,7 @@ class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
             closing tag).
         """
         # TODO: handle namespaces here?
-        attr_dict: Dict[str, str] = {}
+        attr_dict:Dict[str, str] = {}
         for key, value in attrs:
             # Change None attribute values to the empty string
             # for consistency with the other tree builders.
diff --git a/bs4/builder/_lxml.py b/bs4/builder/_lxml.py
index 3dfe88a..380164e 100644
--- a/bs4/builder/_lxml.py
+++ b/bs4/builder/_lxml.py
@@ -13,6 +13,7 @@ from collections.abc import Callable
 
 from typing import (
     Any,
+    cast,
     Dict,
     IO,
     Iterable,
@@ -21,6 +22,7 @@ from typing import (
     Set,
     Tuple,
     Type,
+    TypeAlias,
     TYPE_CHECKING,
     Union,
 )
@@ -28,7 +30,6 @@ from typing import (
 from io import BytesIO
 from io import StringIO
 from lxml import etree
-from bs4.dammit import (_Encoding)
 from bs4.element import (
     Comment,
     Doctype,
@@ -60,13 +61,16 @@ if TYPE_CHECKING:
 
 LXML:str = 'lxml'
 
-def _invert(d):
+def _invert(d:dict[Any, Any]) -> dict[Any, Any]:
     "Invert a dictionary."
     return dict((v,k) for k, v in list(d.items()))
 
+_LXMLParser:TypeAlias = Union[etree.XMLParser, etree.HTMLParser]
+_ParserOrParserClass:TypeAlias = Union[_LXMLParser, Type[etree.XMLParser], Type[etree.HTMLParser]]
+
 class LXMLTreeBuilderForXML(TreeBuilder):
 
-    DEFAULT_PARSER_CLASS:Type[Any] = etree.XMLParser
+    DEFAULT_PARSER_CLASS:Type[etree.XMLParser] = etree.XMLParser
     
     is_xml:bool = True
         
@@ -93,6 +97,7 @@ class LXMLTreeBuilderForXML(TreeBuilder):
     nsmaps: List[Optional[_InvertedNamespaceMapping]]
     empty_element_tags: Set[str]
     parser: Any
+    _default_parser: Optional[etree.XMLParser]
     
     # NOTE: If we parsed Element objects and looked at .sourceline,
     # we'd be able to see the line numbers from the original document.
@@ -137,7 +142,7 @@ class LXMLTreeBuilderForXML(TreeBuilder):
                 # prefix, the first one in the document takes precedence.
                 self.soup._namespaces[key] = value
                 
-    def default_parser(self, encoding:Optional[_Encoding]) -> Type:
+    def default_parser(self, encoding:Optional[_Encoding]) -> _ParserOrParserClass:
         """Find the default parser for the given encoding.
 
         :return: Either a parser object or a class, which
@@ -148,7 +153,7 @@ class LXMLTreeBuilderForXML(TreeBuilder):
         return self.DEFAULT_PARSER_CLASS(
             target=self, strip_cdata=False, recover=True, encoding=encoding)
 
-    def parser_for(self, encoding: Optional[_Encoding]) -> Any:
+    def parser_for(self, encoding: Optional[_Encoding]) -> _LXMLParser:
         """Instantiate an appropriate parser for the given encoding.
 
         :param encoding: A string.
@@ -164,8 +169,8 @@ class LXMLTreeBuilderForXML(TreeBuilder):
             )
         return parser
 
-    def __init__(self, parser:Optional[Any]=None,
-                 empty_element_tags:Optional[Set[str]]=None, **kwargs):
+    def __init__(self, parser:Optional[etree.XMLParser]=None,
+                 empty_element_tags:Optional[Set[str]]=None, **kwargs:Any):
         # TODO: Issue a warning if parser is present but not a
         # callable, since that means there's no way to create new
         # parsers for different encodings.
@@ -270,7 +275,7 @@ class LXMLTreeBuilderForXML(TreeBuilder):
             yield (detector.markup, encoding, document_declared_encoding, False)
 
     def feed(self, markup:_RawMarkup) -> None:
-        io: IO
+        io: Union[BytesIO, StringIO]
         if isinstance(markup, bytes):
             io = BytesIO(markup)
         elif isinstance(markup, str):
@@ -298,14 +303,25 @@ class LXMLTreeBuilderForXML(TreeBuilder):
     def close(self) -> None:
         self.nsmaps = [self.DEFAULT_NSMAPS_INVERTED]
 
-    def start(self, name:str, attrs:Dict[str, str], nsmap:_NamespaceMapping={}):
+    def start(self, tag:str|bytes, attrs:Dict[str|bytes, str|bytes], nsmap:_NamespaceMapping={}) -> None:
         # This is called by lxml code as a result of calling
         # BeautifulSoup.feed(), and we know self.soup is set by the time feed()
         # is called.
         assert self.soup is not None
-        
-        # Make sure attrs is a mutable dict--lxml may send an immutable dictproxy.
-        attrs = dict(attrs)
+        assert isinstance(tag, str)
+
+        # We need to recreate the attribute dict for three
+        # reasons. First, for type checking, so we can assert there
+        # are no bytestrings in the keys or values. Second, because we
+        # need a mutable dict--lxml might send us an immutable
+        # dictproxy. Third, so we can handle namespaced attribute
+        # names by converting the keys to NamespacedAttributes.
+        new_attrs:Dict[Union[str,NamespacedAttribute], str] = {}
+        for k, v in attrs.items():
+            assert isinstance(k, str)
+            assert isinstance(v, str)
+            new_attrs[k] = v
+
         nsprefix: Optional[_NamespacePrefix] = None
         namespace: Optional[_NamespaceURL] = None
         # Invert each namespace map as it comes in.
@@ -340,30 +356,28 @@ class LXMLTreeBuilderForXML(TreeBuilder):
             
             # Also treat the namespace mapping as a set of attributes on the
             # tag, so we can recreate it later.
-            attrs = attrs.copy()
             for prefix, namespace in list(nsmap.items()):
                 attribute = NamespacedAttribute(
                     "xmlns", prefix, "http://www.w3.org/2000/xmlns/")
-                attrs[attribute] = namespace
+                new_attrs[attribute] = namespace
 
         # Namespaces are in play. Find any attributes that came in
         # from lxml with namespaces attached to their names, and
         # turn then into NamespacedAttribute objects.
-        new_attrs:Dict[Union[str,NamespacedAttribute], str] = {}
-        for attr, value in list(attrs.items()):
+        final_attrs:Dict[Union[str,NamespacedAttribute], str] = {}
+        for attr, value in list(new_attrs.items()):
             namespace, attr = self._getNsTag(attr)
             if namespace is None:
-                new_attrs[attr] = value
+                final_attrs[attr] = value
             else:
                 nsprefix = self._prefix_for_namespace(namespace)
                 attr = NamespacedAttribute(nsprefix, attr, namespace)
-                new_attrs[attr] = value
-        attrs = new_attrs
+                final_attrs[attr] = value
 
-        namespace, name = self._getNsTag(name)
+        namespace, tag = self._getNsTag(tag)
         nsprefix = self._prefix_for_namespace(namespace)
         self.soup.handle_starttag(
-            name, namespace, nsprefix, attrs,
+            tag, namespace, nsprefix, final_attrs,
             namespaces=self.active_namespace_prefixes[-1]
         )
         
@@ -376,8 +390,9 @@ class LXMLTreeBuilderForXML(TreeBuilder):
                 return inverted_nsmap[namespace]
         return None
 
-    def end(self, name:str) -> None:
+    def end(self, name:str|bytes) -> None:
         assert self.soup is not None
+        assert isinstance(name, str)
         self.soup.endData()
         completed_tag = self.soup.tagStack[-1]
         namespace, name = self._getNsTag(name)
@@ -406,9 +421,10 @@ class LXMLTreeBuilderForXML(TreeBuilder):
         self.soup.handle_data(data)
         self.soup.endData(self.processing_instruction_class)
         
-    def data(self, content:str) -> None:
+    def data(self, data:str|bytes) -> None:
         assert self.soup is not None
-        self.soup.handle_data(content)
+        assert isinstance(data, str)
+        self.soup.handle_data(data)
 
     def doctype(self, name:str, pubid:str, system:str) -> None:
         assert self.soup is not None
@@ -416,11 +432,12 @@ class LXMLTreeBuilderForXML(TreeBuilder):
         doctype = Doctype.for_name_and_ids(name, pubid, system)
         self.soup.object_was_parsed(doctype)
 
-    def comment(self, content:str) -> None:
+    def comment(self, text:str|bytes) -> None:
         "Handle comments as Comment objects."
         assert self.soup is not None
+        assert isinstance(text, str)
         self.soup.endData()
-        self.soup.handle_data(content)
+        self.soup.handle_data(text)
         self.soup.endData(Comment)
 
     def test_fragment_to_document(self, fragment:str) -> str:
@@ -436,7 +453,7 @@ class LXMLTreeBuilder(HTMLTreeBuilder, LXMLTreeBuilderForXML):
     features: Iterable[str] = list(ALTERNATE_NAMES) + [NAME, HTML, FAST, PERMISSIVE]
     is_xml: bool = False
 
-    def default_parser(self, encoding:Optional[_Encoding]) -> Type[Any]:
+    def default_parser(self, encoding:Optional[_Encoding]) -> _ParserOrParserClass:
         return etree.HTMLParser
 
     def feed(self, markup:_RawMarkup) -> None: 
diff --git a/bs4/dammit.py b/bs4/dammit.py
index 8c1b631..4e950a2 100644
--- a/bs4/dammit.py
+++ b/bs4/dammit.py
@@ -260,14 +260,14 @@ class EntitySubstitution(object):
     AMPERSAND_OR_BRACKET: Pattern[str] = re.compile("([<>&])")
 
     @classmethod
-    def _substitute_html_entity(cls, matchobj:re.Match[str]) -> str:
+    def _substitute_html_entity(cls, matchobj:re.Match) -> str:
         """Used with a regular expression to substitute the
         appropriate HTML entity for a special character string."""
         entity = cls.CHARACTER_TO_HTML_ENTITY.get(matchobj.group(0))
         return "&%s;" % entity
 
     @classmethod
-    def _substitute_xml_entity(cls, matchobj:re.Match[str]) -> str:
+    def _substitute_xml_entity(cls, matchobj:re.Match) -> str:
         """Used with a regular expression to substitute the
         appropriate XML entity for a special character string."""
         entity = cls.CHARACTER_TO_XML_ENTITY[matchobj.group(0)]
@@ -752,7 +752,7 @@ class UnicodeDammit:
 
     log: Logger #: :meta private:
             
-    def _sub_ms_char(self, match:re.Match[bytes]) -> bytes:
+    def _sub_ms_char(self, match:re.Match) -> bytes:
         """Changes a MS smart quote character to an XML or HTML
         entity, or an ASCII character.
 
diff --git a/bs4/element.py b/bs4/element.py
index f4ab89c..22115a2 100644
--- a/bs4/element.py
+++ b/bs4/element.py
@@ -39,11 +39,13 @@ from typing import (
     Union,
     cast,
 )
-from typing_extensions import Self
+from typing_extensions import (
+    Self,
+    TypeAlias,
+)
 if TYPE_CHECKING:
     from bs4 import BeautifulSoup
     from bs4.builder import TreeBuilder
-    from bs4.dammit import _Encoding
     from bs4.filter import ElementFilter
     from bs4.formatter import (
         _EntitySubstitutionFunction,
@@ -52,12 +54,16 @@ if TYPE_CHECKING:
     from bs4._typing import (
         _AttributeValue,
         _AttributeValues,
+        _Encoding,
+        _RawOrProcessedAttributeValues,
         _StrainableElement,
         _StrainableAttribute,
         _StrainableAttributes,
         _StrainableString,
     )
 
+_OneOrMoreStringTypes:TypeAlias = Union[Type['NavigableString'], Iterable[Type['NavigableString']]]
+
 # Deprecated module-level attributes.
 # See https://peps.python.org/pep-0562/
 _deprecated_names = dict(
@@ -66,7 +72,7 @@ _deprecated_names = dict(
 #: :meta private:
 _deprecated_whitespace_re: Pattern[str] = re.compile(r"\s+")
 
-def __getattr__(name):
+def __getattr__(name:str) -> Any:
     if name in _deprecated_names:
         message = _deprecated_names[name]
         warnings.warn(
@@ -124,7 +130,8 @@ class NamespacedAttribute(str):
     namespace: Optional[str]
     
     def __new__(cls, prefix:Optional[str],
-                name:Optional[str]=None, namespace:Optional[str]=None):
+                name:Optional[str]=None,
+                namespace:Optional[str]=None) -> Self:
         if not name:
             # This is the default namespace. Its name "has no value"
             # per https://www.w3.org/TR/xml-names/#defaulting
@@ -223,7 +230,7 @@ class ContentMetaAttributeValue(AttributeValueWithCharsetSubstitution):
         """
         if eventual_encoding in PYTHON_SPECIFIC_ENCODINGS:
             return self.CHARSET_RE.sub('', self.original_value)
-        def rewrite(match):
+        def rewrite(match:re.Match[str]) -> str:
             return match.group(1) + eventual_encoding
         return self.CHARSET_RE.sub(rewrite, self.original_value)
 
@@ -370,7 +377,7 @@ class PageElement(object):
         "previousSibling", "previous_sibling", "4.0.0"
     )
 
-    def __deepcopy__(self, memo:Dict, recursive:bool=False) -> Self:
+    def __deepcopy__(self, memo:Dict[Any,Any], recursive:bool=False) -> Self:
         raise NotImplementedError()
         
     def __copy__(self) -> Self:
@@ -528,6 +535,13 @@ class PageElement(object):
     ) -> Optional[PageElement]:
         """Finds the last element beneath this object to be parsed.
 
+        Special note to help you figure things out if your type
+        checking is tripped up by the fact that this method returns
+        Optional[PageElement] instead of PageElement: the only time
+        this method returns None is if `accept_self` is False and the
+        `PageElement` has no children--either it's a NavigableString
+        or an empty Tag.
+
         :param is_initialized: Has `PageElement.setup` been called on
             this `PageElement` yet?
 
@@ -878,9 +892,10 @@ class PageElement(object):
 
     def _find_one(
             self,
-            # TODO: "There is no syntax to indicate optional or keyword
-            # arguments; such function types are rarely used as
-            # callback types." - So, not sure how to get more specific here.
+            # TODO-TYPING: "There is no syntax to indicate optional or
+            # keyword arguments; such function types are rarely used
+            # as callback types." - So, not sure how to get more
+            # specific here.
             method:Callable,
             name:Optional[_StrainableElement],
             attrs:_StrainableAttributes,
@@ -955,7 +970,7 @@ class PageElement(object):
         You can pass in your own technique for iterating over the tree, and your own
         technique for matching items.
         """
-        results:ResultSet = ResultSet(matcher)
+        results:ResultSet[PageElement] = ResultSet(matcher)
         while True:
             try:
                 i = next(generator)
@@ -1029,27 +1044,27 @@ class PageElement(object):
         return getattr(self, '_decomposed', False) or False
    
     @_deprecated("next_elements", "4.0.0")
-    def nextGenerator(self):
+    def nextGenerator(self) -> Iterator[PageElement]:
         ":meta private:"
         return self.next_elements
 
     @_deprecated("next_siblings", "4.0.0")
-    def nextSiblingGenerator(self):
+    def nextSiblingGenerator(self) -> Iterator[PageElement]:
         ":meta private:"
         return self.next_siblings
 
     @_deprecated("previous_elements", "4.0.0")
-    def previousGenerator(self):
+    def previousGenerator(self) -> Iterator[PageElement]:
         ":meta private:"
         return self.previous_elements
 
     @_deprecated("previous_siblings", "4.0.0")
-    def previousSiblingGenerator(self):
+    def previousSiblingGenerator(self) -> Iterator[PageElement]:
         ":meta private:"
         return self.previous_siblings
 
     @_deprecated("parents", "4.0.0")
-    def parentGenerator(self):
+    def parentGenerator(self) -> Iterator[PageElement]:
         ":meta private:"
         return self.parents
 
@@ -1087,7 +1102,7 @@ class NavigableString(str, PageElement):
         u.setup()
         return u
 
-    def __deepcopy__(self, memo:Dict, recursive:bool=False) -> Self:
+    def __deepcopy__(self, memo:Dict[Any, Any], recursive:bool=False) -> Self:
         """A copy of a NavigableString has the same contents and class
         as the original, but it is not connected to the parse tree.
 
@@ -1097,7 +1112,7 @@ class NavigableString(str, PageElement):
         """
         return type(self)(self)
 
-    def __getnewargs__(self):
+    def __getnewargs__(self) -> Tuple[str]:
         return (str(self),)
 
     @property
@@ -1134,14 +1149,14 @@ class NavigableString(str, PageElement):
         return None
 
     @name.setter
-    def name(self, name:str):
+    def name(self, name:str) -> None:
         """Prevent NavigableString.name from ever being set.
 
         :meta private:
         """
         raise AttributeError("A NavigableString cannot be given a name.")
 
-    def _all_strings(self, strip=False, types:Iterable[Type[NavigableString]]=PageElement.default) -> Iterator[str]:
+    def _all_strings(self, strip:bool=False, types:_OneOrMoreStringTypes=PageElement.default) -> Iterator[str]:
         """Yield all strings of certain classes, possibly stripping them.
 
         This makes it easy for NavigableString to implement methods
@@ -1382,7 +1397,7 @@ class Tag(PageElement):
                  name:Optional[str]=None,
                  namespace:Optional[str]=None,
                  prefix:Optional[str]=None,
-                 attrs:Optional[_AttributeValues]=None,
+                 attrs:Optional[_RawOrProcessedAttributeValues]=None,
                  parent:Optional[Union[BeautifulSoup, Tag]]=None,
                  previous:Optional[PageElement]=None,
                  is_xml:Optional[bool]=None,
@@ -1485,7 +1500,7 @@ class Tag(PageElement):
     #: :meta private:
     parserClass = _deprecated_alias("parserClass", "parser_class", "4.0.0")
 
-    def __deepcopy__(self, memo:dict, recursive:bool=True) -> Self:
+    def __deepcopy__(self, memo:Dict[Any, Any], recursive:bool=True) -> Self:
         """A deepcopy of a Tag is a new Tag, unconnected to the parse tree.
         Its contents are a copy of the old Tag's contents.
         """
@@ -1552,9 +1567,9 @@ class Tag(PageElement):
         return len(self.contents) == 0 and self.can_be_empty_element is True
 
     @_deprecated("is_empty_element", "4.0.0")
-    def isSelfClosing(self):
+    def isSelfClosing(self) -> bool:
         ": :meta private:"
-        return is_empty_element()
+        return self.is_empty_element
 
     @property
     def string(self) -> Optional[str]:
@@ -1592,7 +1607,7 @@ class Tag(PageElement):
 
     #: :meta private:
     MAIN_CONTENT_STRING_TYPES = {NavigableString, CData}
-    def _all_strings(self, strip:bool=False, types:Iterable[Type[NavigableString]]=PageElement.default) -> Iterator[str]:
+    def _all_strings(self, strip:bool=False, types:_OneOrMoreStringTypes=PageElement.default) -> Iterator[str]:
         """Yield all strings of certain classes, possibly stripping them.
 
         :param strip: If True, all strings will be stripped before being
@@ -1739,7 +1754,7 @@ class Tag(PageElement):
     replace_with_children = unwrap
 
     @_deprecated("unwrap", "4.0.0")
-    def replaceWithChildren(self):
+    def replaceWithChildren(self) -> PageElement:
         ": :meta private:"
         return self.unwrap()
     
@@ -1914,11 +1929,21 @@ class Tag(PageElement):
         "Deleting tag[key] deletes all 'key' attributes for the tag."
         self.attrs.pop(key, None)
 
-    def __call__(self, *args, **kwargs) -> ResultSet[PageElement]:
+    def __call__(self,
+            name:Optional[_StrainableElement]=None,
+            attrs:_StrainableAttributes={},
+            recursive:bool=True,
+            string:Optional[_StrainableString]=None,
+            limit:Optional[int]=None,
+            _stacklevel:int=2,
+            **kwargs:_StrainableAttribute
+        )-> ResultSet[PageElement]:
         """Calling a Tag like a function is the same as calling its
         find_all() method. Eg. tag('a') returns a list of all the A tags
         found within this tag."""
-        return self.find_all(*args, **kwargs)
+        return self.find_all(
+            name, attrs, recursive, string, limit, _stacklevel, **kwargs
+        )
 
     def __getattr__(self, subtag:str) -> Optional[Tag]:
         """Calling tag.subtag is the same as calling tag.find(name="subtag")"""
@@ -2002,7 +2027,7 @@ class Tag(PageElement):
     def decode(self, indent_level:Optional[int]=None,
                eventual_encoding:_Encoding=DEFAULT_OUTPUT_ENCODING,
                formatter:_FormatterOrName="minimal",
-               iterator:Optional[Iterable]=None) -> str:
+               iterator:Optional[Iterator[PageElement]]=None) -> str:
         """Render this `Tag` and its contents as a Unicode string.
 
         :param indent_level: Each line of the rendering will be
@@ -2122,9 +2147,10 @@ class Tag(PageElement):
     EMPTY_ELEMENT_EVENT = _TreeTraversalEvent() #: :meta private:
     STRING_ELEMENT_EVENT = _TreeTraversalEvent() #: :meta private:
 
-    def _event_stream(self, iterator=None) -> Iterator[
-            Tuple[_TreeTraversalEvent, PageElement]
-    ]:
+    def _event_stream(
+            self,
+            iterator:Optional[Iterator[PageElement]]=None
+    ) -> Iterator[Tuple[_TreeTraversalEvent, PageElement]]:
         """Yield a sequence of events that can be used to reconstruct the DOM
         for this element.
 
@@ -2316,8 +2342,8 @@ class Tag(PageElement):
         return contents.encode(encoding)
 
     @_deprecated("encode_contents", "4.0.0")
-    def renderContents(self, encoding=DEFAULT_OUTPUT_ENCODING,
-                       prettyPrint=False, indentLevel=0):
+    def renderContents(self, encoding:str=DEFAULT_OUTPUT_ENCODING,
+                       prettyPrint:bool=False, indentLevel:Optional[int]=0) -> bytes:
         """Deprecated method for BS3 compatibility.
 
         :meta private:
@@ -2436,7 +2462,7 @@ class Tag(PageElement):
     def select_one(self,
                    selector:str,
                    namespaces:Optional[Dict[str, str]]=None,
-                   **kwargs) -> Optional[Tag]:
+                   **kwargs:Any) -> Optional[Tag]:
         """Perform a CSS selection operation on the current element.
 
         :param selector: A CSS selector.
@@ -2452,7 +2478,7 @@ class Tag(PageElement):
         return self.css.select_one(selector, namespaces, **kwargs)
 
     def select(self, selector:str, namespaces:Optional[Dict[str, str]]=None,
-               limit:int=0, **kwargs) -> ResultSet[Tag]:
+               limit:int=0, **kwargs:Any) -> ResultSet[Tag]:
         """Perform a CSS selection operation on the current element.
 
         This uses the SoupSieve library.
@@ -2478,7 +2504,7 @@ class Tag(PageElement):
 
     # Old names for backwards compatibility
     @_deprecated("children", "4.0.0")
-    def childGenerator(self):
+    def childGenerator(self) -> Iterator[PageElement]:
         """Deprecated generator.
 
         :meta private:
@@ -2486,7 +2512,7 @@ class Tag(PageElement):
         return self.children
 
     @_deprecated("descendants", "4.0.0")
-    def recursiveChildGenerator(self):
+    def recursiveChildGenerator(self) -> Iterator[PageElement]:
         """Deprecated generator.
 
         :meta private:
@@ -2494,7 +2520,7 @@ class Tag(PageElement):
         return self.descendants
 
     @_deprecated("has_attr", "4.0.0")
-    def has_key(self, key):
+    def has_key(self, key:str) -> bool:
         """Deprecated method. This was kind of misleading because has_key()
         (attributes) was different from __in__ (contents).
 
@@ -2516,7 +2542,7 @@ class ResultSet(List[_PageElementT], Generic[_PageElementT]):
         super(ResultSet, self).__init__(result)
         self.source = source
 
-    def __getattr__(self, key:str):
+    def __getattr__(self, key:str) -> None:
         """Raise a helpful exception to explain a common code fix."""
         raise AttributeError(
             f"""ResultSet object has no attribute "{key}". You're probably treating a list of elements like a single element. Did you call find_all() when you meant to call find()?"""
diff --git a/bs4/filter.py b/bs4/filter.py
index 74e26d9..7632639 100644
--- a/bs4/filter.py
+++ b/bs4/filter.py
@@ -25,10 +25,10 @@ from bs4._deprecation import _deprecated
 from bs4.element import NavigableString, PageElement, Tag
 from bs4._typing import (
     _AttributeValue,
-    _AttributeValues,
     _AllowStringCreationFunction,
     _AllowTagCreationFunction,
     _PageElementMatchFunction,
+    _RawAttributeValues,
     _TagMatchFunction,
     _StringMatchFunction,
     _StrainableElement,
@@ -98,7 +98,7 @@ class ElementFilter(object):
 
     def allow_tag_creation(
             self, nsprefix:Optional[str], name:str,
-            attrs:Optional[_AttributeValues]
+            attrs:Optional[_RawAttributeValues]
     ) -> bool:
         """Based on the name and attributes of a tag, see whether this
         ElementFilter will allow a Tag object to even be created.
@@ -372,9 +372,8 @@ class SoupStrainer(ElementFilter):
             # third-party regex library, whose pattern objects doesn't
             # derive from re.Pattern.
             #
-            # TODO-TYPING: Once we drop support for Python 3.7, we
-            # might be able to address this by defining an appropriate
-            # Protocol.
+            # TODO-TYPING: We should be able to bring in a Protocol
+            # from typing_extensions to handle this.
             yield rule_class(pattern=obj)
         elif hasattr(obj, '__iter__'):
             for o in obj:
@@ -497,7 +496,7 @@ class SoupStrainer(ElementFilter):
             )
         return this_attr_match
 
-    def allow_tag_creation(self, nsprefix:Optional[str], name:str, attrs:Optional[_AttributeValues]) -> bool:
+    def allow_tag_creation(self, nsprefix:Optional[str], name:str, attrs:Optional[_RawAttributeValues]) -> bool:
         """Based on the name and attributes of a tag, see whether this
         SoupStrainer will allow a Tag object to even be created.
 
@@ -586,7 +585,7 @@ class SoupStrainer(ElementFilter):
         return False
 
     @_deprecated("allow_tag_creation", "4.13.0")
-    def search_tag(self, name:str, attrs:Optional[_AttributeValues]) -> bool:
+    def search_tag(self, name:str, attrs:Optional[_RawAttributeValues]) -> bool:
         """A less elegant version of allow_tag_creation()."""
         ":meta private:"
         return self.allow_tag_creation(None, name, attrs)
diff --git a/bs4/tests/__init__.py b/bs4/tests/__init__.py
index 3ef999d..8415645 100644
--- a/bs4/tests/__init__.py
+++ b/bs4/tests/__init__.py
@@ -15,6 +15,7 @@ from bs4.element import (
     Comment,
     ContentMetaAttributeValue,
     Doctype,
+    PageElement,
     PYTHON_SPECIFIC_ENCODINGS,
     Script,
     Stylesheet,
@@ -25,8 +26,21 @@ from bs4.builder import (
     DetectsXMLParsedAsHTML,
     XMLParsedAsHTMLWarning,
 )
+from bs4._typing import (
+    _IncomingMarkup
+)
+
+from bs4.builder import TreeBuilder
 from bs4.builder._htmlparser import HTMLParserTreeBuilder
-default_builder = HTMLParserTreeBuilder
+
+from typing import (
+    Any,
+    Iterable,
+    List,
+    Optional,
+    Tuple,
+    Type,
+)
 
 # Some tests depend on specific third-party libraries. We use
 # @pytest.mark.skipIf on the following conditionals to skip them
@@ -51,7 +65,9 @@ except ImportError:
     LXML_PRESENT = False
     LXML_VERSION = (0,)
 
-BAD_DOCUMENT = """A bare string
+default_builder:Type[TreeBuilder] = HTMLParserTreeBuilder
+
+BAD_DOCUMENT:str = """A bare string
 <!DOCTYPE xsl:stylesheet SYSTEM "htmlent.dtd">
 <!DOCTYPE xsl:stylesheet PUBLIC "htmlent.dtd">
 <div><![CDATA[A CDATA section where it doesn't belong]]></div>
@@ -91,28 +107,30 @@ BAD_DOCUMENT = """A bare string
 class SoupTest(object):
 
     @property
-    def default_builder(self):
+    def default_builder(self) -> Type[TreeBuilder]:
         return default_builder
 
-    def soup(self, markup, **kwargs):
+    def soup(self, markup:_IncomingMarkup, **kwargs:Any) -> BeautifulSoup:
         """Build a Beautiful Soup object from markup."""
         builder = kwargs.pop('builder', self.default_builder)
         return BeautifulSoup(markup, builder=builder, **kwargs)
 
-    def document_for(self, markup, **kwargs):
+    def document_for(self, markup:str, **kwargs:Any) -> str:
         """Turn an HTML fragment into a document.
 
         The details depend on the builder.
         """
         return self.default_builder(**kwargs).test_fragment_to_document(markup)
    
-    def assert_soup(self, to_parse, compare_parsed_to=None):
+    def assert_soup(self, to_parse:_IncomingMarkup,
+                    compare_parsed_to:Optional[str]=None) -> None:
         """Parse some markup using Beautiful Soup and verify that
         the output markup is as expected.
         """
         builder = self.default_builder
         obj = BeautifulSoup(to_parse, builder=builder)
         if compare_parsed_to is None:
+            assert isinstance(to_parse, str)
             compare_parsed_to = to_parse
 
         # Verify that the documents come out the same.
@@ -131,7 +149,7 @@ class SoupTest(object):
 
     assertSoupEquals = assert_soup
         
-    def assertConnectedness(self, element):
+    def assertConnectedness(self, element:Tag) -> None:
         """Ensure that next_element and previous_element are properly
         set for all descendants of the given element.
         """
@@ -142,7 +160,7 @@ class SoupTest(object):
                 assert earlier == e.previous_element
             earlier = e
 
-    def linkage_validator(self, el, _recursive_call=False):
+    def linkage_validator(self, el:Tag, _recursive_call:bool=False) -> Optional[PageElement]:
         """Ensure proper linkage throughout the document."""
         descendant = None
         # Document element should have no previous element or previous sibling.
@@ -209,6 +227,7 @@ class SoupTest(object):
 
             if isinstance(child, Tag) and child.contents:
                 descendant = self.linkage_validator(child, True)
+                assert descendant is not None
                 # A bubbled up descendant should have no next siblings
                 assert descendant.next_sibling is None,\
                     "Bad next_sibling\nNODE: {}\nNEXT {}\nEXPECTED {}".format(
@@ -234,7 +253,7 @@ class SoupTest(object):
             child = el
 
         if not _recursive_call and child is not None:
-            target = el
+            target:Optional[Tag] = el
             while True:
                 if target is None:
                     assert child.next_element is None, \
@@ -256,7 +275,7 @@ class SoupTest(object):
             # Return the child to the recursive caller
             return child
 
-    def assert_selects(self, tags, should_match):
+    def assert_selects(self, tags:Iterable[Tag], should_match:Iterable[str]) -> None:
         """Make sure that the given tags have the correct text.
 
         This is used in tests that define a bunch of tags, each
@@ -265,7 +284,7 @@ class SoupTest(object):
         """
         assert [tag.string for tag in tags] == should_match
 
-    def assert_selects_ids(self, tags, should_match):
+    def assert_selects_ids(self, tags:Iterable[Tag], should_match:Iterable[str]) -> None:
         """Make sure that the given tags have the correct IDs.
 
         This is used in tests that define a bunch of tags, each
@@ -275,7 +294,7 @@ class SoupTest(object):
         assert [tag['id'] for tag in tags] == should_match
 
 
-class TreeBuilderSmokeTest(object):
+class TreeBuilderSmokeTest(SoupTest):
     # Tests that are common to HTML and XML tree builders.
 
     @pytest.mark.parametrize(
@@ -352,7 +371,7 @@ class HTMLTreeBuilderSmokeTest(TreeBuilderSmokeTest):
         assert loaded.__class__ == BeautifulSoup
         assert loaded.decode() == tree.decode()
 
-    def assertDoctypeHandled(self, doctype_fragment):
+    def assertDoctypeHandled(self, doctype_fragment:str) -> None:
         """Assert that a given doctype string is handled correctly."""
         doctype_str, soup = self._document_with_doctype(doctype_fragment)
 
@@ -366,7 +385,7 @@ class HTMLTreeBuilderSmokeTest(TreeBuilderSmokeTest):
         # parse tree and that the rest of the document parsed.
         assert soup.p.contents[0] == 'foo'
 
-    def _document_with_doctype(self, doctype_fragment, doctype_string="DOCTYPE"):
+    def _document_with_doctype(self, doctype_fragment:str, doctype_string:str="DOCTYPE") -> Tuple[bytes, BeautifulSoup]:
         """Generate and parse a document with the given doctype."""
         doctype = '<!%s %s>' % (doctype_string, doctype_fragment)
         markup = doctype + '\n<p>foo</p>'
diff --git a/bs4/tests/test_builder_registry.py b/bs4/tests/test_builder_registry.py
index 9a9ce1f..da10e5f 100644
--- a/bs4/tests/test_builder_registry.py
+++ b/bs4/tests/test_builder_registry.py
@@ -2,10 +2,12 @@
 
 import pytest
 import warnings
+from typing import Type
 
 from bs4 import BeautifulSoup
 from bs4.builder import (
     builder_registry as registry,
+    TreeBuilder,
     TreeBuilderRegistry,
 )
 from bs4.builder._htmlparser import HTMLParserTreeBuilder
@@ -81,7 +83,7 @@ class TestRegistry(object):
     def setup_method(self):
         self.registry = TreeBuilderRegistry()
 
-    def builder_for_features(self, *feature_list):
+    def builder_for_features(self, *feature_list:str) -> Type[TreeBuilder]:
         cls = type('Builder_' + '_'.join(feature_list),
                    (object,), {'features' : feature_list})
 
diff --git a/bs4/tests/test_css.py b/bs4/tests/test_css.py
index 359dbcd..2e2baba 100644
--- a/bs4/tests/test_css.py
+++ b/bs4/tests/test_css.py
@@ -8,6 +8,12 @@ from bs4 import (
     ResultSet,
 )
 
+from typing import (
+    Any,
+    Iterable,
+    Tuple,
+)
+
 from . import (
     SoupTest,
     SOUP_SIEVE_PRESENT,
@@ -78,7 +84,7 @@ class TestCSSSelectors(SoupTest):
     def setup_method(self):
         self.soup = BeautifulSoup(self.HTML, 'html.parser')
 
-    def assert_selects(self, selector, expected_ids, **kwargs):
+    def assert_selects(self, selector:str, expected_ids:Iterable[str], **kwargs:Any) -> None:
         results = self.soup.select(selector, **kwargs)
         assert isinstance(results, ResultSet)
         el_ids = [el['id'] for el in results]
@@ -90,7 +96,7 @@ class TestCSSSelectors(SoupTest):
 
     assertSelect = assert_selects
 
-    def assert_select_multiple(self, *tests):
+    def assert_select_multiple(self, *tests:Tuple[str, Iterable[str]]):
         for selector, expected_ids in tests:
             self.assert_selects(selector, expected_ids)
 
diff --git a/bs4/tests/test_filter.py b/bs4/tests/test_filter.py
index 8d5da70..dfd6f18 100644
--- a/bs4/tests/test_filter.py
+++ b/bs4/tests/test_filter.py
@@ -5,6 +5,12 @@ import warnings
 from . import (
     SoupTest,
 )
+from typing import (
+    Callable,
+    Optional,
+    Pattern,
+    Tuple,
+)
 from bs4.element import Tag
 from bs4.filter import (
     AttributeValueMatchRule,
@@ -14,6 +20,8 @@ from bs4.filter import (
     StringMatchRule,
     TagNameMatchRule,
 )
+from bs4._typing import _RawOrProcessedAttributeValues
+
 
 class TestElementFilter(SoupTest):
 
@@ -107,7 +115,10 @@ class TestElementFilter(SoupTest):
 
 class TestMatchRule(SoupTest):
 
-    def _tuple(self, rule):
+    def _tuple(self, rule:MatchRule) -> Tuple[Optional[str],
+                                              Optional[Pattern[str]],
+                                              Optional[Callable],
+                                              Optional[bool]]:
         return (
             rule.string,
             rule.pattern.pattern if rule.pattern else None,
@@ -395,9 +406,10 @@ class TestSoupStrainer(SoupTest):
             assert msg == "Ignoring nested list [[...]] to avoid the possibility of infinite recursion."
 
     def tag_matches(
-            self, strainer, name, attrs=None, string=None, prefix=None,
-            match_valence=True
-    ):
+            self, strainer:SoupStrainer, name:str,
+            attrs:Optional[_RawOrProcessedAttributeValues]=None,
+            string:Optional[str]=None, prefix:Optional[str]=None,
+    ) -> bool:
         # Create a Tag with the given prefix, name and attributes,
         # then make sure that strainer.matches_tag and allow_tag_creation
         # both approve it.
diff --git a/bs4/tests/test_fuzz.py b/bs4/tests/test_fuzz.py
index f29802d..579686d 100644
--- a/bs4/tests/test_fuzz.py
+++ b/bs4/tests/test_fuzz.py
@@ -38,7 +38,7 @@ class TestFuzz(object):
     # multiple copies of the code must be kept around to run against
     # older tests. I'm not sure what to do about this, but I may
     # retire old tests after a time.
-    def fuzz_test_with_css(self, filename):
+    def fuzz_test_with_css(self, filename:str) -> None:
         data = self.__markup(filename)
         parsers = ['lxml-xml', 'html5lib', 'html.parser', 'lxml']
         try:
@@ -168,7 +168,7 @@ class TestFuzz(object):
     def test_html5lib_parse_errors(self, filename):
         self.fuzz_test_with_css(filename)
         
-    def __markup(self, filename):
+    def __markup(self, filename:str) -> bytes:
         if not filename.endswith(self.TESTCASE_SUFFIX):
             filename += self.TESTCASE_SUFFIX
         this_dir = os.path.split(__file__)[0]
diff --git a/bs4/tests/test_html5lib.py b/bs4/tests/test_html5lib.py
index 9f6dfa1..3c34403 100644
--- a/bs4/tests/test_html5lib.py
+++ b/bs4/tests/test_html5lib.py
@@ -8,14 +8,13 @@ from bs4.filter import SoupStrainer
 from . import (
     HTML5LIB_PRESENT,
     HTML5TreeBuilderSmokeTest,
-    SoupTest,
 )
 
 @pytest.mark.skipif(
     not HTML5LIB_PRESENT,
     reason="html5lib seems not to be present, not testing its tree builder."
 )
-class TestHTML5LibBuilder(SoupTest, HTML5TreeBuilderSmokeTest):
+class TestHTML5LibBuilder(HTML5TreeBuilderSmokeTest):
     """See ``HTML5TreeBuilderSmokeTest``."""
 
     @property
diff --git a/bs4/tests/test_htmlparser.py b/bs4/tests/test_htmlparser.py
index ff0f305..2a13c99 100644
--- a/bs4/tests/test_htmlparser.py
+++ b/bs4/tests/test_htmlparser.py
@@ -10,12 +10,14 @@ from bs4.builder import (
     XMLParsedAsHTMLWarning,
 )
 from bs4.builder._htmlparser import (
+    _DuplicateAttributeHandler,
     BeautifulSoupHTMLParser,
     HTMLParserTreeBuilder,
 )
+from typing import Any
 from . import SoupTest, HTMLTreeBuilderSmokeTest
 
-class TestHTMLParserTreeBuilder(SoupTest, HTMLTreeBuilderSmokeTest):
+class TestHTMLParserTreeBuilder(HTMLTreeBuilderSmokeTest):
 
     default_builder = HTMLParserTreeBuilder
 
@@ -95,7 +97,7 @@ class TestHTMLParserTreeBuilder(SoupTest, HTMLTreeBuilderSmokeTest):
         assert "id" == soup.a['id']
         
         # You can also get this behavior explicitly.
-        def assert_attribute(on_duplicate_attribute, expected):
+        def assert_attribute(on_duplicate_attribute:_DuplicateAttributeHandler, expected:Any) -> None:
             soup = self.soup(
                 markup, on_duplicate_attribute=on_duplicate_attribute
             )
diff --git a/bs4/tests/test_lxml.py b/bs4/tests/test_lxml.py
index 9fc04e0..dc20501 100644
--- a/bs4/tests/test_lxml.py
+++ b/bs4/tests/test_lxml.py
@@ -26,7 +26,7 @@ from . import (
     not LXML_PRESENT,
     reason="lxml seems not to be present, not testing its tree builder."
 )
-class TestLXMLTreeBuilder(SoupTest, HTMLTreeBuilderSmokeTest):
+class TestLXMLTreeBuilder(HTMLTreeBuilderSmokeTest):
     """See ``HTMLTreeBuilderSmokeTest``."""
 
     @property
@@ -88,7 +88,7 @@ class TestLXMLTreeBuilder(SoupTest, HTMLTreeBuilderSmokeTest):
     not LXML_PRESENT,
     reason="lxml seems not to be present, not testing its XML tree builder."
 )
-class TestLXMLXMLTreeBuilder(SoupTest, XMLTreeBuilderSmokeTest):
+class TestLXMLXMLTreeBuilder(XMLTreeBuilderSmokeTest):
     """See ``HTMLTreeBuilderSmokeTest``."""
 
     @property
diff --git a/bs4/tests/test_soup.py b/bs4/tests/test_soup.py
index c95f380..61d0235 100644
--- a/bs4/tests/test_soup.py
+++ b/bs4/tests/test_soup.py
@@ -8,6 +8,7 @@ import pickle
 import pytest
 import sys
 import tempfile
+from typing import Iterable
 
 from bs4 import (
     BeautifulSoup,
@@ -260,14 +261,15 @@ class TestWarnings(SoupTest):
     # that the code that triggered the warning is in the same
     # file as the test.
 
-    def _assert_warning(self, warnings, cls):
+    def _assert_warning(
+            self, warnings:Iterable[Warning], cls:type[Warning]) -> Warning:
         for w in warnings:
             if isinstance(w.message, cls):
                 assert w.filename == __file__
                 return w
         raise Exception("%s warning not found in %r" % (cls, warnings))
     
-    def _assert_no_parser_specified(self, w):
+    def _assert_no_parser_specified(self, w:Warning) -> None:
         warning = self._assert_warning(w, GuessedAtParserWarning)
         message = str(warning.message)
         assert message.startswith(BeautifulSoup.NO_PARSER_SPECIFIED_WARNING[:60])
diff --git a/bs4/tests/test_tree.py b/bs4/tests/test_tree.py
index 43afb29..fe56e6b 100644
--- a/bs4/tests/test_tree.py
+++ b/bs4/tests/test_tree.py
@@ -135,7 +135,7 @@ class TestFindAllBasicNamespaces(SoupTest):
 class TestFindAllByName(SoupTest):
     """Test ways of finding tags by tag name."""
 
-    def setup_method(self):
+    def setup_method(self) -> None:
         self.tree =  self.soup("""<a>First tag.</a>
                                   <b>Second tag.</b>
                                   <c>Third <a>Nested tag.</a> tag.</c>""")
@@ -459,7 +459,7 @@ class TestIndex(SoupTest):
 class TestParentOperations(SoupTest):
     """Test navigation and searching through an element's parents."""
 
-    def setup_method(self):
+    def setup_method(self) -> None:
         self.tree = self.soup('''<ul id="empty"></ul>
                                  <ul id="top">
                                   <ul id="middle">
@@ -508,14 +508,14 @@ class TestParentOperations(SoupTest):
 
 class ProximityTest(SoupTest):
 
-    def setup_method(self):
+    def setup_method(self) -> None:
         self.tree = self.soup(
             '<html id="start"><head></head><body><b id="1">One</b><b id="2">Two</b><b id="3">Three</b></body></html>')
 
 
 class TestNextOperations(ProximityTest):
 
-    def setup_method(self):
+    def setup_method(self) -> None:
         super(TestNextOperations, self).setup_method()
         self.start = self.tree.b
 
@@ -555,7 +555,7 @@ class TestNextOperations(ProximityTest):
 
 class TestPreviousOperations(ProximityTest):
 
-    def setup_method(self):
+    def setup_method(self) -> None:
         super(TestPreviousOperations, self).setup_method()
         self.end = self.tree.find(string="Three")
 
@@ -604,7 +604,7 @@ class TestPreviousOperations(ProximityTest):
 
 class SiblingTest(SoupTest):
 
-    def setup_method(self):
+    def setup_method(self) -> None:
         markup = '''<html>
                     <span id="1">
                      <span id="1.1"></span>
@@ -625,7 +625,7 @@ class SiblingTest(SoupTest):
 
 class TestNextSibling(SiblingTest):
 
-    def setup_method(self):
+    def setup_method(self) -> None:
         super(TestNextSibling, self).setup_method()
         self.start = self.tree.find(id="1")
 
@@ -670,7 +670,7 @@ class TestNextSibling(SiblingTest):
 
 class TestPreviousSibling(SiblingTest):
 
-    def setup_method(self):
+    def setup_method(self) -> None:
         super(TestPreviousSibling, self).setup_method()
         self.end = self.tree.find(id="4")
 
diff --git a/doc/index.rst b/doc/index.rst
index a414830..0ff1fb2 100755
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -3029,38 +3029,64 @@ Advanced search techniques
 
 Almost everyone who uses Beautiful Soup to extract information from a
 document can get what they need using the methods described in
-`Searching the tree`_. However, there's a lower-level interface--the
-:py:class:`ElementSelector` class-- which lets you define any matching
+`Searching the tree`_. However, there is a lower-level interface--the
+:py:class:`ElementFilter` class--that lets you define any matching
 behavior whatsoever.
 
-To use :py:class:`ElementSelector`, define a function that takes a
-:py:class:`PageElement` object (that is, it might be either a
-:py:class:`Tag` or a :py:class`NavigableString`) and returns ``True``
+To use :py:class:`ElementFilter`, define a function that takes a
+:py:class:`PageElement` object (which can be either a :py:class:`Tag`
+*or* :py:class:`NavigableString` object) and returns ``True``
 (if the element matches your custom criteria) or ``False`` (if it
-doesn't)::
+does not)::
 
-  [example goes here]
+ def _match_non_whitespace(pe):
+     """
+     return True for:
+     * all Tag objects
+     * NavigableString objects that contain non-whitespace text
+     """
+     return (
+         isinstance(pe, Tag) or
+         (isinstance(pe, NavigableString) and
+             pe.text and not pe.text.isspace()))
 
-Then, pass the function into an :py:class:`ElementSelector`::
+Then, construct an :py:class:`ElementFilter` that uses your function::
 
- from bs4.select import ElementSelector
- selector = ElementSelector(f)
+ from bs4.filter import ElementFilter
+ skip_whitespace = ElementFilter(match_function=_match_non_whitespace)
 
-You can then pass the :py:class:`ElementSelector` object as the first
+You can now pass this :py:class:`ElementFilter` object as the first
 argument to any of the `Searching the tree`_ methods::
 
- [examples go here]
+ from bs4 import BeautifulSoup
+ html_doc = """
+ <p>
+   <b>bold</b>
+   <i>italic</i>
+   and
+   <u>underline</u>
+ </p>
+ """
+ soup = BeautifulSoup(html_doc, 'lxml')
 
-Every potential match will be run through your function, and the only
-:py:class:`PageElement` objects returned will be the one where your
+ soup.find('p').find_all(skip_whitespace, recursive=False)
+ # [<b>bold</b>, <i>italic</i>, '\n  and\n  ', <u>underline</u>]
+
+Every :py:class:`PageElement` encountered will be evaluated by your
+function, and the objects returned will be only the ones where your
 function returned ``True``.
 
-Note that this is different from simply passing `a function`_ as the
-first argument to one of the search methods. That's an easy way to
-find a tag, but _only_ tags will be considered. With an
-:py:class:`ElementSelector` you can write a single function that makes
-decisions about both tags and strings.
- 
+To summarize the function-based matching behaviors,
+
+* A function passed as the first argument to a search method
+  (or equivalently, using the ``name`` argument) considers only
+  :py:class:`Tag` objects.
+* A function passed to a search method using the ``string`` argument
+  considers only :py:class:`NavigableString` objects.
+* A function passed to a search method using an :py:class:`ElementFilter`
+  object considers both :py:class:`Tag` and :py:class:`NavigableString`
+  objects.
+
 
 Advanced parser customization
 =============================