Merge ~chrispitude/beautifulsoup:node-filters into beautifulsoup:master

[Chris Papademetrious]

implement "node" filtering that considers all PageElement objects

This is a draft merge request that implements a proof-of-concept for the following wishlist request:

#2047713: enhance find*() methods to filter through all object types
https://bugs.launchpad.net/beautifulsoup/+bug/2047713

Most of the changes are to thread a new "node" filter down into the machinery. The actual functionality is just three additional lines in the search() method to use it.

With these changes, when the following script is run:

====
#!/bin/env python
from bs4 import BeautifulSoup, NavigableString
html_doc = """
  <p>
    <b>bold</b>
    <i>italic</i>
    <u>underline</u>
    text
    <br />
  </p>
"""
soup = BeautifulSoup(html_doc, 'lxml')

# this is the filter I want to use
def is_non_whitespace(thing) -> bool:
    return not (isinstance(thing, NavigableString) and thing.text.isspace())

# get the first non-whitespace thing in <p>
this_thing = soup.find('p').find(node=is_non_whitespace, recursive=False)

# print all following non-whitespace sibling elements in <p>
while this_thing:
    next_thing = this_thing.find_next_sibling(node=is_non_whitespace)
    print(f"{repr(this_thing)} is followed by {repr(next_thing)}")
    this_thing = next_thing
====

the results are as follows:

====
<b>bold</b> is followed by <i>italic</i>
<i>italic</i> is followed by <u>underline</u>
<u>underline</u> is followed by '\n text\n '
'\n text\n ' is followed by <br/>
<br/> is followed by None
====

Note the mix of tag and text objects!

Some questions and open items:

* Is "PageElement" the correct term for any object in a BeautifulSoup document (Tag, NavigableString, Comment, ProcessingInstruction, etc.)?
* What should this new filter argument be called? (node? page_element? something else?)
* Is there a more elegant approach that doesn't require threading a new argument down into everything?
* Rules for mixing this new filter with existing name/attribute filters must be defined/coded/documented.
  * I think this new filter should be mutually exclusive with tag/attribute filters.
  * I think this new filter should accept only Callable objects, and perhaps also True/False.
* Tests and documentation are needed.
  * I can do this when the implementation is complete.

Fingers crossed that this makes it in. It would be enormously powerful.

[Chris Papademetrious (chrispitude)]

I will redo this merge proposal on top of the 4.13 branch.

Before I start, are there any aspects that you'd like to see implemented differently?

a771557... by Chris Papademetrious on 2024-01-01

implement a filter that considers all PageElement objects

Unmerged commits

a771557... by Chris Papademetrious on 2024-01-01

implement a filter that considers all PageElement objects

8a6d1dd... by Leonard Richardson on 2023-06-04

Merged in change to main branch.

4cde600... by Leonard Richardson on 2023-05-12

Those casts are more trouble than they're worth.

1113a86... by Leonard Richardson on 2023-05-12

Got css.py to pass mypy strict although it's a little hacky.

26e1772... by Leonard Richardson on 2023-05-12

Went through formatter.py with mypy strict.

5bf3787... by Leonard Richardson on 2023-05-12

Went through dammit.py with mypy strict.

7200655... by Leonard Richardson on 2023-05-12

Merged in main branch.

f3a3619... by Leonard Richardson on 2023-05-12

Got rid of deprecation warnings in tests.

6f89323... by Leonard Richardson on 2023-05-08

Get (slightly) more specific about alias.

f8e55c0... by Leonard Richardson on 2023-04-18

_alias itself is not used anywhere.

diff --git a/CHANGELOG b/CHANGELOG
index 66fcb74..bec1e11 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,3 +1,102 @@
+= 4.13.0 (Unreleased)
+
+* This version drops support for Python 3.6. The minimum supported
+  major version for Beautiful Soup is now Python 3.7.
+
+* Deprecation warnings have been added for all deprecated methods and
+  attributes. Most of these were deprecated over ten years ago, and
+  some were deprecated over fifteen years ago.
+
+  Going forward, deprecated names will be subject to removal two
+  feature releases or one major release after the deprecation warning
+  is added.
+
+* append(), extend(), insert(), and unwrap() were moved from PageElement to
+  Tag. Those methods manipulate the 'contents' collection, so they would
+  only have ever worked on Tag objects.
+
+* decompose() was moved from Tag to PageElement, since there's no reason
+  it won't also work on NavigableString objects.
+
+* The BeautifulSoupHTMLParser constructor now requires a BeautifulSoup
+  object as its first argument. This almost certainly does not affect
+  you, since you probably use HTMLParserTreeBuilder, not
+  BeautifulSoupHTMLParser directly.
+
+* If Tag.get_attribute_list() is used to access an attribute that's not set,
+  the return value is now an empty list rather than [None].
+
+* AttributeValueWithCharsetSubstitution.encode() is renamed to
+  substitute_encoding, to avoid confusion with the much different str.encode()
+
+* Using PageElement.replace_with() to replace an element with itself
+  returns the element instead of None.
+
+* When using one of the find() methods or creating a SoupStrainer,
+  if you specify the same attribute value in ``attrs`` and the
+  keyword arguments, you'll end up with two different ways to match that
+  attribute. Previously the value in keyword arguments would override the
+  value in ``attrs``.
+
+* When using one of the find() methods or creating a SoupStrainer, you can
+  pass a list of any accepted object (strings, regular expressions, etc.) for
+  any of the objects. Previously you could only pass in a list of strings.
+
+* A SoupStrainer can now filter tag creation based on a tag's
+  namespaced name. Previously only the unqualified name could be used.
+
+* All TreeBuilder constructors now take the empty_element_tags
+  argument. The sets of tags found in HTMLTreeBuilder.empty_element_tags and
+  HTMLTreeBuilder.block_elements are now in
+  HTMLTreeBuilder.DEFAULT_EMPTY_ELEMENT_TAGS and
+  HTMLTreeBuilder.DEFAULT_BLOCK_ELEMENTS, to avoid confusing them with
+  instance variables.
+
+* Issue a warning if a document is parsed using a SoupStrainer that's just
+  going to filter everything. In these cases, filtering everything is the
+  most consistent thing to do, but there was no indication that was
+  happening.
+
+* UnicodeDammit.markup is now always a bytestring representing the
+  *original* markup (sans BOM), and UnicodeDammit.unicode_markup is
+  always the same markup, converted to Unicode. Previously,
+  UnicodeDammit.markup was treated inconsistently and would often end
+  up containing Unicode. UnicodeDammit.markup was not a documented
+  attribute, but if you were using it, you probably want to switch to using
+  .unicode_markup instead.
+
+* Corrected the markup that's output in the unlikely event that you
+  encode a document to a Python internal encoding (like "palmos")
+  that's not recognized by the HTML or XML standard.
+
+* The arguments to LXMLTreeBuilderForXML.prepare_markup have been
+  changed to match the arguments to the superclass,
+  TreeBuilder.prepare_markup. Specifically, document_declared_encoding
+  now appears before exclude_encodings, not after. If you were calling
+  this method yourself, I recomment switching to using keyword
+  arguments instead.
+
+* Fixed an error in the lookup table used when converting
+  ISO-Latin-1 to ASCII, which no one should do anyway.
+
+* The unused constant LXMLTreeBuilderForXML.DEFAULT_PARSER_CLASS
+  has been removed.
+
+New deprecations in 4.13.0:
+
+* The SAXTreeBuilder class, which was never officially supported or tested.
+
+* The first argument to BeautifulSoup.decode has been changed from a bool
+  `pretty_print` to an int `indent_level`, to match the signature of Tag.decode.
+
+* SoupStrainer.text and SoupStrainer.string are both deprecated
+  since a single item can't capture all the possibilities of a SoupStrainer
+  designed to match strings.
+
+* SoupStrainer.search_tag() is deprecated. It was never a
+  documented method, but if you use it, you should start using
+  SoupStrainer.allow_tag_creation() instead.
+
 = 4.12.3 (?)
 
 * Fixed a regression such that if you set .hidden on a tag, the tag
diff --git a/bs4/__init__.py b/bs4/__init__.py
index 3d2ab09..a1289c7 100644
--- a/bs4/__init__.py
+++ b/bs4/__init__.py
@@ -7,8 +7,8 @@ Beautiful Soup uses a pluggable XML or HTML parser to parse a
 provides methods and Pythonic idioms that make it easy to navigate,
 search, and modify the parse tree.
 
-Beautiful Soup works with Python 3.6 and up. It works better if lxml
-and/or html5lib is installed.
+Beautiful Soup works with Python 3.7 and up. It works better if lxml
+and/or html5lib is installed, but they are not required.
 
 For more than you ever wanted to know about Beautiful Soup, see the
 documentation: http://www.crummy.com/software/BeautifulSoup/bs4/doc/
@@ -37,9 +37,10 @@ if sys.version_info.major < 3:
 from .builder import (
     builder_registry,
     ParserRejectedMarkup,
+    TreeBuilder,
     XMLParsedAsHTMLWarning,
-    HTMLParserTreeBuilder
 )
+from .builder._htmlparser import HTMLParserTreeBuilder
 from .dammit import UnicodeDammit
 from .element import (
     CData,
@@ -55,10 +56,32 @@ from .element import (
     ResultSet,
     Script,
     Stylesheet,
-    SoupStrainer,
     Tag,
     TemplateString,
     )
+from .formatter import Formatter
+from .strainer import SoupStrainer
+from typing import (
+    Any,
+    cast,
+    Counter as CounterType,
+    Dict,
+    Iterable,
+    List,
+    Sequence,
+    Optional,
+    Type,
+    TYPE_CHECKING,
+    Union,
+)
+
+from bs4._typing import (
+    _AttributeValue,
+    _AttributeValues,
+    _Encoding,
+    _Encodings,
+    _IncomingMarkup,
+)
 
 # Define some custom warnings.
 class GuessedAtParserWarning(UserWarning):
@@ -104,24 +127,64 @@ class BeautifulSoup(Tag):
     handle_endtag.
     """
 
-    # Since BeautifulSoup subclasses Tag, it's possible to treat it as
-    # a Tag with a .name. This name makes it clear the BeautifulSoup
-    # object isn't a real markup tag.
-    ROOT_TAG_NAME = '[document]'
-
-    # If the end-user gives no indication which tree builder they
-    # want, look for one with these features.
-    DEFAULT_BUILDER_FEATURES = ['html', 'fast']
-
-    # A string containing all ASCII whitespace characters, used in
-    # endData() to detect data chunks that seem 'empty'.
-    ASCII_SPACES = '\x20\x0a\x09\x0c\x0d'
-
-    NO_PARSER_SPECIFIED_WARNING = "No parser was explicitly specified, so I'm using the best available %(markup_type)s parser for this system (\"%(parser)s\"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.\n\nThe code that caused this warning is on line %(line_number)s of the file %(filename)s. To get rid of this warning, pass the additional argument 'features=\"%(parser)s\"' to the BeautifulSoup constructor.\n"
-   
-    def __init__(self, markup="", features=None, builder=None,
-                 parse_only=None, from_encoding=None, exclude_encodings=None,
-                 element_classes=None, **kwargs):
+    #: Since `BeautifulSoup` subclasses `Tag`, it's possible to treat it as
+    #: a `Tag` with a `Tag.name`. Hoever, this name makes it clear the
+    #: `BeautifulSoup` object isn't a real markup tag.
+    ROOT_TAG_NAME:str = '[document]'
+
+    #: If the end-user gives no indication which tree builder they
+    #: want, look for one with these features.
+    DEFAULT_BUILDER_FEATURES: Sequence[str] = ['html', 'fast']
+
+    #: A string containing all ASCII whitespace characters, used in
+    #: `BeautifulSoup.endData` to detect data chunks that seem 'empty'.
+    ASCII_SPACES: str = '\x20\x0a\x09\x0c\x0d'
+
+    #: :meta private:
+    NO_PARSER_SPECIFIED_WARNING: str = "No parser was explicitly specified, so I'm using the best available %(markup_type)s parser for this system (\"%(parser)s\"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.\n\nThe code that caused this warning is on line %(line_number)s of the file %(filename)s. To get rid of this warning, pass the additional argument 'features=\"%(parser)s\"' to the BeautifulSoup constructor.\n"
+
+    # FUTURE PYTHON:
+    element_classes:Dict[Type[PageElement], Type[Any]] #: :meta private:
+    builder:TreeBuilder #: :meta private:
+    is_xml: bool
+    known_xml: Optional[bool]
+    parse_only: Optional[SoupStrainer] #: :meta private:
+
+    # These members are only used while parsing markup.
+    markup:Optional[Union[str,bytes]] #: :meta private:
+    current_data:List[str] #: :meta private:
+    currentTag:Optional[Tag] #: :meta private:
+    tagStack:List[Tag] #: :meta private:
+    open_tag_counter:CounterType[str] #: :meta private:
+    preserve_whitespace_tag_stack:List[Tag] #: :meta private:
+    string_container_stack:List[Tag] #: :meta private:
+
+    #: Beautiful Soup's best guess as to the character encoding of the
+    #: original document.
+    original_encoding: Optional[_Encoding]
+
+    #: The character encoding, if any, that was explicitly defined
+    #: in the original document. This may or may not match
+    #: `BeautifulSoup.original_encoding`.
+    declared_html_encoding: Optional[_Encoding]
+    
+    #: This is True if the markup that was parsed contains
+    #: U+FFFD REPLACEMENT_CHARACTER characters which were not present
+    #: in the original markup. These mark character sequences that
+    #: could not be represented in Unicode.
+    contains_replacement_characters: bool
+    
+    def __init__(
+            self,
+            markup:_IncomingMarkup="",
+            features:Optional[Union[str,Sequence[str]]]=None,
+            builder:Optional[Union[TreeBuilder,Type[TreeBuilder]]]=None,
+            parse_only:Optional[SoupStrainer]=None,
+            from_encoding:Optional[_Encoding]=None,
+            exclude_encodings:Optional[_Encodings]=None,
+            element_classes:Optional[Dict[Type[PageElement], Type[Any]]]=None,
+            **kwargs:Any
+    ):
         """Constructor.
 
         :param markup: A string or a file-like object representing
@@ -196,14 +259,14 @@ class BeautifulSoup(Tag):
         if 'selfClosingTags' in kwargs:
             del kwargs['selfClosingTags']
             warnings.warn(
-                "BS4 does not respect the selfClosingTags argument to the "
+                "Beautiful Soup 4 does not respect the selfClosingTags argument to the "
                 "BeautifulSoup constructor. The tree builder is responsible "
                 "for understanding self-closing tags.")
 
         if 'isHTML' in kwargs:
             del kwargs['isHTML']
             warnings.warn(
-                "BS4 does not respect the isHTML argument to the "
+                "Beautiful Soup 4 does not respect the isHTML argument to the "
                 "BeautifulSoup constructor. Suggest you use "
                 "features='lxml' for HTML and features='lxml-xml' for "
                 "XML.")
@@ -212,7 +275,8 @@ class BeautifulSoup(Tag):
             if old_name in kwargs:
                 warnings.warn(
                     'The "%s" argument to the BeautifulSoup constructor '
-                    'has been renamed to "%s."' % (old_name, new_name),
+                    'was renamed to "%s" in Beautiful Soup 4.0.0' % (
+                        old_name, new_name),
                     DeprecationWarning, stacklevel=3
                 )
                 return kwargs.pop(old_name)
@@ -220,7 +284,14 @@ class BeautifulSoup(Tag):
 
         parse_only = parse_only or deprecated_argument(
             "parseOnlyThese", "parse_only")
-
+        if (parse_only is not None
+            and parse_only.string_rules and
+            (parse_only.name_rules or parse_only.attribute_rules)):
+            warnings.warn(
+                f"Value for parse_only will exclude everything, since it puts restrictions on both tags and strings: {parse_only}",
+                UserWarning, stacklevel=3
+            )
+        
         from_encoding = from_encoding or deprecated_argument(
             "fromEncoding", "from_encoding")
 
@@ -235,7 +306,8 @@ class BeautifulSoup(Tag):
         # specify a parser' warning.
         original_builder = builder
         original_features = features
-            
+
+        builder_class: Type[TreeBuilder]
         if isinstance(builder, type):
             # A builder class was passed in; it needs to be instantiated.
             builder_class = builder
@@ -245,12 +317,13 @@ class BeautifulSoup(Tag):
                 features = [features]
             if features is None or len(features) == 0:
                 features = self.DEFAULT_BUILDER_FEATURES
-            builder_class = builder_registry.lookup(*features)
-            if builder_class is None:
+            possible_builder_class = builder_registry.lookup(*features)
+            if possible_builder_class is None:
                 raise FeatureNotFound(
                     "Couldn't find a tree builder with the features you "
                     "requested: %s. Do you need to install a parser library?"
                     % ",".join(features))
+            builder_class = cast(Type[TreeBuilder], possible_builder_class)
 
         # At this point either we have a TreeBuilder instance in
         # builder, or we have a builder_class that we can instantiate
@@ -259,7 +332,8 @@ class BeautifulSoup(Tag):
             builder = builder_class(**kwargs)
             if not original_builder and not (
                     original_features == builder.NAME or
-                    original_features in builder.ALTERNATE_NAMES
+                    (isinstance(original_features, str)
+                     and original_features in builder.ALTERNATE_NAMES)
             ) and markup:
                 # The user did not tell us which TreeBuilder to use,
                 # and we had to guess. Issue a warning.
@@ -323,6 +397,10 @@ class BeautifulSoup(Tag):
             if not self._markup_is_url(markup):
                 self._markup_resembles_filename(markup)                
 
+        # At this point we know markup is a string or bytestring.  If
+        # it was a file-type object, we've read from it.
+        markup = cast(Union[str,bytes], markup)
+                
         rejections = []
         success = False
         for (self.markup, self.original_encoding, self.declared_html_encoding,
@@ -486,7 +564,7 @@ class BeautifulSoup(Tag):
         markup.
         """
         Tag.__init__(self, self, self.builder, self.ROOT_TAG_NAME)
-        self.hidden = 1
+        self.hidden = True
         self.builder.reset()
         self.current_data = []
         self.currentTag = None
@@ -497,8 +575,16 @@ class BeautifulSoup(Tag):
         self._most_recent_element = None
         self.pushTag(self)
 
-    def new_tag(self, name, namespace=None, nsprefix=None, attrs={},
-                sourceline=None, sourcepos=None, **kwattrs):
+    def new_tag(
+            self,
+            name:str,
+            namespace:Optional[str]=None,
+            nsprefix:Optional[str]=None,
+            attrs:_AttributeValues={},
+            sourceline:Optional[int]=None,
+            sourcepos:Optional[int]=None,
+            **kwattrs:_AttributeValue,
+    ):
         """Create a new Tag associated with this BeautifulSoup object.
 
         :param name: The name of the new Tag.
@@ -509,7 +595,7 @@ class BeautifulSoup(Tag):
             that are reserved words in Python.
         :param sourceline: The line number where this tag was
             (purportedly) found in its source document.
-        :param sourcepos: The character position within `sourceline` where this
+        :param sourcepos: The character position within ``sourceline`` where this
             tag was (purportedly) found.
         :param kwattrs: Keyword arguments for the new Tag's attribute values.
 
@@ -520,9 +606,17 @@ class BeautifulSoup(Tag):
             sourceline=sourceline, sourcepos=sourcepos
         )
 
-    def string_container(self, base_class=None):
+    def string_container(self,
+                         base_class:Optional[Type[NavigableString]]=None
+                         ) -> Type[NavigableString]:
+        """Find the class that should be instantiated to hold a given kind of
+        string.
+
+        This may be a built-in Beautiful Soup class or a custom class passed
+        in to the BeautifulSoup constructor.
+        """
         container = base_class or NavigableString
-        
+
         # There may be a general override of NavigableString.
         container = self.element_classes.get(
             container, container
@@ -536,27 +630,40 @@ class BeautifulSoup(Tag):
             )
         return container
         
-    def new_string(self, s, subclass=None):
-        """Create a new NavigableString associated with this BeautifulSoup
+    def new_string(self, s:str, subclass:Optional[Type[NavigableString]]=None) -> NavigableString:
+        """Create a new `NavigableString` associated with this `BeautifulSoup`
         object.
+
+        :param s: The string content of the `NavigableString`
+
+        :param subclass: The subclass of `NavigableString`, if any, to
+        use. If a document is being processed, an appropriate subclass
+        for the current location in the document will be determined
+        automatically.
         """
         container = self.string_container(subclass)
         return container(s)
 
-    def insert_before(self, *args):
+    def insert_before(self, *args:PageElement) -> None:
         """This method is part of the PageElement API, but `BeautifulSoup` doesn't implement
         it because there is nothing before or after it in the parse tree.
         """
         raise NotImplementedError("BeautifulSoup objects don't support insert_before().")
 
-    def insert_after(self, *args):
+    def insert_after(self, *args:PageElement) -> None:
         """This method is part of the PageElement API, but `BeautifulSoup` doesn't implement
         it because there is nothing before or after it in the parse tree.
         """
         raise NotImplementedError("BeautifulSoup objects don't support insert_after().")
 
-    def popTag(self):
-        """Internal method called by _popToTag when a tag is closed."""
+    def popTag(self) -> Optional[Tag]:
+        """Internal method called by _popToTag when a tag is closed.
+
+        :meta private:
+        """
+        if not self.tagStack:
+            # Nothing to pop. This shouldn't happen.
+            return None
         tag = self.tagStack.pop()
         if tag.name in self.open_tag_counter:
             self.open_tag_counter[tag.name] -= 1
@@ -569,8 +676,11 @@ class BeautifulSoup(Tag):
             self.currentTag = self.tagStack[-1]
         return self.currentTag
 
-    def pushTag(self, tag):
-        """Internal method called by handle_starttag when a tag is opened."""
+    def pushTag(self, tag:Tag) -> None:
+        """Internal method called by handle_starttag when a tag is opened.
+
+        :meta private:
+        """
         #print("Push", tag.name)
         if self.currentTag is not None:
             self.currentTag.contents.append(tag)
@@ -583,9 +693,14 @@ class BeautifulSoup(Tag):
         if tag.name in self.builder.string_containers:
             self.string_container_stack.append(tag)
 
-    def endData(self, containerClass=None):
+    def endData(self, containerClass:Optional[Type[NavigableString]]=None) -> None:
         """Method called by the TreeBuilder when the end of a data segment
         occurs.
+
+        :param containerClass: The class to use when incorporating the
+        data segment into the parse tree.
+
+        :meta private:
         """       
         if self.current_data:
             current_data = ''.join(self.current_data)
@@ -609,18 +724,27 @@ class BeautifulSoup(Tag):
 
             # Should we add this string to the tree at all?
             if self.parse_only and len(self.tagStack) <= 1 and \
-                   (not self.parse_only.text or \
-                    not self.parse_only.search(current_data)):
+                   (not self.parse_only.string_rules or \
+                    not self.parse_only.allow_string_creation(current_data)):
                 return
 
             containerClass = self.string_container(containerClass)
             o = containerClass(current_data)
             self.object_was_parsed(o)
 
-    def object_was_parsed(self, o, parent=None, most_recent_element=None):
-        """Method called by the TreeBuilder to integrate an object into the parse tree."""
+    def object_was_parsed(
+            self, o:PageElement, parent:Optional[Tag]=None,
+            most_recent_element:Optional[PageElement]=None):
+        """Method called by the TreeBuilder to integrate an object into the
+        parse tree.
+
+        
+
+        :meta private:
+        """
         if parent is None:
             parent = self.currentTag
+        assert parent is not None
         if most_recent_element is not None:
             previous_element = most_recent_element
         else:
@@ -685,7 +809,7 @@ class BeautifulSoup(Tag):
                 break
             target = target.parent
 
-    def _popToTag(self, name, nsprefix=None, inclusivePop=True):
+    def _popToTag(self, name, nsprefix=None, inclusivePop=True) -> Optional[Tag]:
         """Pops the tag stack up to and including the most recent
         instance of the given tag.
 
@@ -698,11 +822,12 @@ class BeautifulSoup(Tag):
           to but *not* including the most recent instqance of the
           given tag.
 
+        :meta private:
         """
         #print("Popping to %s" % name)
         if name == self.ROOT_TAG_NAME:
             # The BeautifulSoup object itself can never be popped.
-            return
+            return None
 
         most_recently_popped = None
 
@@ -719,8 +844,11 @@ class BeautifulSoup(Tag):
 
         return most_recently_popped
 
-    def handle_starttag(self, name, namespace, nsprefix, attrs, sourceline=None,
-                        sourcepos=None, namespaces=None):
+    def handle_starttag(
+            self, name:str, namespace:Optional[str],
+            nsprefix:Optional[str], attrs:Optional[Dict[str,str]],
+            sourceline:Optional[int]=None, sourcepos:Optional[int]=None,
+            namespaces:Optional[Dict[str, str]]=None) -> Optional[Tag]:
         """Called by the tree builder when a new tag is encountered.
 
         :param name: Name of the tag.
@@ -737,13 +865,15 @@ class BeautifulSoup(Tag):
         SoupStrainer. You should proceed as if the tag had not occurred
         in the document. For instance, if this was a self-closing tag,
         don't call handle_endtag.
+
+        :meta private:
         """
         # print("Start tag %s: %s" % (name, attrs))
         self.endData()
 
         if (self.parse_only and len(self.tagStack) <= 1
-            and (self.parse_only.text
-                 or not self.parse_only.search_tag(name, attrs))):
+            and (self.parse_only.string_rules
+                 or not self.parse_only.allow_tag_creation(nsprefix, name, attrs))):
             return None
 
         tag = self.element_classes.get(Tag, Tag)(
@@ -760,48 +890,90 @@ class BeautifulSoup(Tag):
         self.pushTag(tag)
         return tag
 
-    def handle_endtag(self, name, nsprefix=None):
+    def handle_endtag(self, name:str, nsprefix:Optional[str]=None) -> None:
         """Called by the tree builder when an ending tag is encountered.
 
         :param name: Name of the tag.
         :param nsprefix: Namespace prefix for the tag.
+
+        :meta private:
         """
         #print("End tag: " + name)
         self.endData()
         self._popToTag(name, nsprefix)
         
-    def handle_data(self, data):
-        """Called by the tree builder when a chunk of textual data is encountered."""
+    def handle_data(self, data:str) -> None:
+        """Called by the tree builder when a chunk of textual data is
+        encountered.
+
+        :meta private:
+        """
         self.current_data.append(data)
        
-    def decode(self, pretty_print=False,
-               eventual_encoding=DEFAULT_OUTPUT_ENCODING,
-               formatter="minimal", iterator=None):
-        """Returns a string or Unicode representation of the parse tree
-            as an HTML or XML document.
-
-        :param pretty_print: If this is True, indentation will be used to
-            make the document more readable.
+    def decode(self, indent_level:Optional[int]=None,
+               eventual_encoding:_Encoding=DEFAULT_OUTPUT_ENCODING,
+               formatter:Union[Formatter,str]="minimal",
+               iterator:Optional[Iterable]=None, **kwargs) -> str:
+        """Returns a string representation of the parse tree
+            as a full HTML or XML document.
+
+        :param indent_level: Each line of the rendering will be
+           indented this many levels. (The ``formatter`` decides what a
+           'level' means, in terms of spaces or other characters
+           output.) This is used internally in recursive calls while
+           pretty-printing.
         :param eventual_encoding: The encoding of the final document.
             If this is None, the document will be a Unicode string.
+        :param formatter: Either a `Formatter` object, or a string naming one of
+            the standard formatters.
+        :param iterator: The iterator to use when navigating over the
+            parse tree. This is only used by `Tag.decode_contents` and
+            you probably won't need to use it.
         """
         if self.is_xml:
             # Print the XML declaration
             encoding_part = ''
+            declared_encoding: Optional[str] = eventual_encoding
             if eventual_encoding in PYTHON_SPECIFIC_ENCODINGS:
                 # This is a special Python encoding; it can't actually
                 # go into an XML document because it means nothing
                 # outside of Python.
-                eventual_encoding = None
-            if eventual_encoding != None:
-                encoding_part = ' encoding="%s"' % eventual_encoding
+                declared_encoding = None
+            if declared_encoding != None:
+                encoding_part = ' encoding="%s"' % declared_encoding
             prefix = '<?xml version="1.0"%s?>\n' % encoding_part
         else:
             prefix = ''
-        if not pretty_print:
-            indent_level = None
+
+        # Prior to 4.13.0, the first argument to this method was a
+        # bool called pretty_print, which gave the method a different
+        # signature from its superclass implementation, Tag.decode.
+        #
+        # The signatures of the two methods now match, but just in
+        # case someone is still passing a boolean in as the first
+        # argument to this method (or a keyword argument with the old
+        # name), we can handle it and put out a DeprecationWarning.
+        warning:Optional[str] = None
+        if isinstance(indent_level, bool):
+            if indent_level is True:
+                indent_level = 0
+            elif indent_level is False:
+                indent_level = None
+            warning = f"As of 4.13.0, the first argument to BeautifulSoup.decode has been changed from bool to int, to match Tag.decode. Pass in a value of {indent_level} instead."
         else:
-            indent_level = 0
+            pretty_print = kwargs.pop("pretty_print", None)
+            assert not kwargs
+            if pretty_print is not None:
+                if pretty_print is True:
+                    indent_level = 0
+                elif pretty_print is False:                
+                    indent_level = None
+                warning = f"As of 4.13.0, the pretty_print argument to BeautifulSoup.decode has been removed, to match Tag.decode. Pass in a value of indent_level={indent_level} instead."
+
+        if warning:
+            warnings.warn(warning, DeprecationWarning, stacklevel=2)
+        elif indent_level is False or pretty_print is False:
+            indent_level = None
         return prefix + super(BeautifulSoup, self).decode(
             indent_level, eventual_encoding, formatter, iterator)
 
@@ -815,7 +987,7 @@ class BeautifulStoneSoup(BeautifulSoup):
     def __init__(self, *args, **kwargs):
         kwargs['features'] = 'xml'
         warnings.warn(
-            'The BeautifulStoneSoup class is deprecated. Instead of using '
+            'The BeautifulStoneSoup class was deprecated in version 4.0.0. Instead of using '
             'it, pass features="xml" into the BeautifulSoup constructor.',
             DeprecationWarning, stacklevel=2
         )
diff --git a/bs4/_deprecation.py b/bs4/_deprecation.py
 new file mode 100644
index 0000000..febc1b3
--- /dev/null
+++ b/bs4/_deprecation.py
@@ -0,0 +1,57 @@
+"""Helper functions for deprecation.
+
+This interface is itself unstable and may change without warning. Do
+not use these functions yourself, even as a joke. The underscores are
+there for a reson.
+
+In particular, most of this will go away once Beautiful Soup drops
+support for Python 3.11, since Python 3.12 defines a
+`@typing.deprecated() decorator. <https://peps.python.org/pep-0702/>`_
+"""
+
+import functools
+import warnings
+
+from typing import (
+    Any,
+    Callable,
+)
+
+def _deprecated_alias(old_name, new_name, version):
+    """Alias one attribute name to another for backward compatibility
+
+    :meta private:
+    """
+    @property
+    def alias(self) -> Any:
+        ":meta private:"
+        warnings.warn(f"Access to deprecated property {old_name}. (Replaced by {new_name}) -- Deprecated since version {version}.", DeprecationWarning, stacklevel=2)
+        return getattr(self, new_name)
+
+    @alias.setter
+    def alias(self, value:str)->Any:
+        ":meta private:"
+        warnings.warn(f"Write to deprecated property {old_name}. (Replaced by {new_name}) -- Deprecated since version {version}.", DeprecationWarning, stacklevel=2)
+        return setattr(self, new_name, value)
+    return alias
+
+def _deprecated_function_alias(old_name:str, new_name:str, version:str) -> Callable:
+    def alias(self, *args, **kwargs):
+        ":meta private:"
+        warnings.warn(f"Call to deprecated method {old_name}. (Replaced by {new_name}) -- Deprecated since version {version}.", DeprecationWarning, stacklevel=2)
+        return getattr(self, new_name)(*args, **kwargs)
+    return alias
+
+def _deprecated(replaced_by:str, version:str) -> Callable:
+    def deprecate(func):
+        @functools.wraps(func)
+        def with_warning(*args, **kwargs):
+            ":meta private:"
+            warnings.warn(
+                f"Call to deprecated method {func.__name__}. (Replaced by {replaced_by}) -- Deprecated since version {version}.",
+                DeprecationWarning,
+                stacklevel=2
+            )
+            return func(*args, **kwargs)
+        return with_warning
+    return deprecate
diff --git a/bs4/_typing.py b/bs4/_typing.py
 new file mode 100644
index 0000000..9fe58c3
--- /dev/null
+++ b/bs4/_typing.py
@@ -0,0 +1,99 @@
+# Custom type aliases used throughout Beautiful Soup to improve readability.
+
+# Notes on improvements to the type system in newer versions of Python
+# that can be used once Beautiful Soup drops support for older
+# versions:
+#
+# * In 3.10, x|y is an accepted shorthand for Union[x,y].
+# * In 3.10, TypeAlias gains capabilities that can be used to
+#   improve the tree matching types (I don't remember what, exactly).
+
+import re
+from typing_extensions import TypeAlias
+from typing import (
+    Callable,
+    Dict,
+    IO,
+    Iterable,
+    Pattern,
+    TYPE_CHECKING,
+    Union,
+)
+
+if TYPE_CHECKING:
+    from bs4.element import Tag
+
+# Aliases for markup in various stages of processing.
+#
+# The rawest form of markup: either a string or an open filehandle.
+_IncomingMarkup: TypeAlias = Union[str,bytes,IO]
+
+# Markup that is in memory but has (potentially) yet to be converted
+# to Unicode.
+_RawMarkup: TypeAlias = Union[str,bytes]
+
+# Aliases for character encodings
+#
+_Encoding:TypeAlias = str
+_Encodings:TypeAlias = Iterable[_Encoding]
+
+# Aliases for XML namespaces
+_NamespacePrefix:TypeAlias = str
+_NamespaceURL:TypeAlias = str
+_NamespaceMapping:TypeAlias = Dict[_NamespacePrefix, _NamespaceURL]
+_InvertedNamespaceMapping:TypeAlias = Dict[_NamespaceURL, _NamespacePrefix]
+
+# Aliases for the attribute values associated with HTML/XML tags.
+#
+# Note that these are attribute values in their final form, as stored
+# in the `Tag` class.  Different parsers present attributes to the
+# `TreeBuilder` subclasses in different formats, which are not defined
+# here.
+_AttributeValue: TypeAlias = Union[str, Iterable[str]]
+_AttributeValues: TypeAlias = Dict[str, _AttributeValue]
+
+# Aliases to represent the many possibilities for matching bits of a
+# parse tree.
+#
+# This is very complicated because we're applying a formal type system
+# to some very DWIM code. The types we end up with will be the types
+# of the arguments to the SoupStrainer constructor and (more
+# familiarly to Beautiful Soup users) the find* methods.
+
+# A function that takes a Tag and returns a yes-or-no answer.
+# A TagNameMatchRule expects this kind of function, if you're
+# going to pass it a function.
+_TagMatchFunction:TypeAlias = Callable[['Tag'], bool]
+
+# A function that takes a single string and returns a yes-or-no
+# answer. An AttributeValueMatchRule expects this kind of function, if
+# you're going to pass it a function. So does a StringMatchRule
+_StringMatchFunction:TypeAlias = Callable[[str], bool]
+
+# A function that takes a Tag or string and returns a yes-or-no
+# answer.
+_TagOrStringMatchFunction:TypeAlias = Union[_TagMatchFunction, _StringMatchFunction, bool]
+
+# Either a tag name, an attribute value or a string can be matched
+# against a string, bytestring, regular expression, or a boolean.
+_BaseStrainable:TypeAlias = Union[str, bytes, Pattern[str], bool]
+
+# A tag can also be matched using a function that takes the Tag
+# as its sole argument.
+_BaseStrainableElement:TypeAlias = Union[_BaseStrainable, _TagMatchFunction]
+
+# A tag's attribute value can be matched using a function that takes
+# the value as its sole argument.
+_BaseStrainableAttribute:TypeAlias = Union[_BaseStrainable, _StringMatchFunction]
+
+# Finally, a tag name, attribute or string can be matched using either
+# a single criterion or a list of criteria.
+_StrainableElement:TypeAlias = Union[
+    _BaseStrainableElement, Iterable[_BaseStrainableElement]
+]
+_StrainableAttribute:TypeAlias = Union[
+    _BaseStrainableAttribute, Iterable[_BaseStrainableAttribute]
+]
+
+_StrainableAttributes:TypeAlias = Dict[str, _StrainableAttribute]
+_StrainableString:TypeAlias = _StrainableAttribute
diff --git a/bs4/builder/__init__.py b/bs4/builder/__init__.py
index 2e39745..671315d 100644
--- a/bs4/builder/__init__.py
+++ b/bs4/builder/__init__.py
@@ -1,9 +1,25 @@
+from __future__ import annotations
 # Use of this source code is governed by the MIT license.
 __license__ = "MIT"
 
 from collections import defaultdict
 import itertools
 import re
+from types import ModuleType
+from typing import (
+    Any,
+    cast,
+    Dict,
+    Iterable,
+    List,
+    Optional,
+    Pattern,
+    Set,
+    Tuple,
+    Type,
+    TYPE_CHECKING,
+    Union,
+)
 import warnings
 import sys
 from bs4.element import (
@@ -17,6 +33,18 @@ from bs4.element import (
     nonwhitespace_re
 )
 
+if TYPE_CHECKING:
+    from bs4 import BeautifulSoup
+    from bs4.element import (
+        NavigableString, Tag,
+        _AttributeValues, _AttributeValue,
+    )
+    from bs4._typing import (
+        _Encoding,
+        _Encodings,
+        _RawMarkup,
+    )
+    
 __all__ = [
     'HTMLTreeBuilder',
     'SAXTreeBuilder',
@@ -36,29 +64,32 @@ class XMLParsedAsHTMLWarning(UserWarning):
     """The warning issued when an HTML parser is used to parse
     XML that is not XHTML.
     """
-    MESSAGE = """It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor."""
+    MESSAGE:str = """It looks like you're parsing an XML document using an HTML parser. If this really is an HTML document (maybe it's XHTML?), you can ignore or filter this warning. If it's XML, you should know that using an XML parser will be more reliable. To parse this document as XML, make sure you have the lxml package installed, and pass the keyword argument `features="xml"` into the BeautifulSoup constructor.""" #: :meta private:
 
 
 class TreeBuilderRegistry(object):
     """A way of looking up TreeBuilder subclasses by their name or by desired
     features.
     """
+
+    builders_for_feature: Dict[str, List[Type[TreeBuilder]]]
+    builders: List[Type[TreeBuilder]]
     
     def __init__(self):
         self.builders_for_feature = defaultdict(list)
         self.builders = []
 
-    def register(self, treebuilder_class):
+    def register(self, treebuilder_class:type[TreeBuilder]) -> None:
         """Register a treebuilder based on its advertised features.
 
-        :param treebuilder_class: A subclass of Treebuilder. its .features
-           attribute should list its features.
+        :param treebuilder_class: A subclass of `Treebuilder`. its
+           `TreeBuilder.features` attribute should list its features.
         """
         for feature in treebuilder_class.features:
             self.builders_for_feature[feature].insert(0, treebuilder_class)
         self.builders.insert(0, treebuilder_class)
 
-    def lookup(self, *features):
+    def lookup(self, *features:str) -> Optional[Type[TreeBuilder]]:
         """Look up a TreeBuilder subclass with the desired features.
 
         :param features: A list of features to look for. If none are
@@ -78,12 +109,12 @@ class TreeBuilderRegistry(object):
 
         # Go down the list of features in order, and eliminate any builders
         # that don't match every feature.
-        features = list(features)
-        features.reverse()
+        feature_list = list(features)
+        feature_list.reverse()
         candidates = None
         candidate_set = None
-        while len(features) > 0:
-            feature = features.pop()
+        while len(feature_list) > 0:
+            feature = feature_list.pop()
             we_have_the_feature = self.builders_for_feature.get(feature, [])
             if len(we_have_the_feature) > 0:
                 if candidates is None:
@@ -97,81 +128,61 @@ class TreeBuilderRegistry(object):
         # The only valid candidates are the ones in candidate_set.
         # Go through the original list of candidates and pick the first one
         # that's in candidate_set.
-        if candidate_set is None:
+        if candidate_set is None or candidates is None:
             return None
         for candidate in candidates:
             if candidate in candidate_set:
                 return candidate
         return None
 
-# The BeautifulSoup class will take feature lists from developers and use them
-# to look up builders in this registry.
-builder_registry = TreeBuilderRegistry()
+#: The `BeautifulSoup` constructor will take a list of features
+#: and use it to look up `TreeBuilder` classes in this registry.
+builder_registry:TreeBuilderRegistry = TreeBuilderRegistry()
 
 class TreeBuilder(object):
-    """Turn a textual document into a Beautiful Soup object tree."""
-
-    NAME = "[Unknown tree builder]"
-    ALTERNATE_NAMES = []
-    features = []
-
-    is_xml = False
-    picklable = False
-    empty_element_tags = None # A tag will be considered an empty-element
-                              # tag when and only when it has no contents.
-    
-    # A value for these tag/attribute combinations is a space- or
-    # comma-separated list of CDATA, rather than a single CDATA.
-    DEFAULT_CDATA_LIST_ATTRIBUTES = defaultdict(list)
-
-    # Whitespace should be preserved inside these tags.
-    DEFAULT_PRESERVE_WHITESPACE_TAGS = set()
-
-    # The textual contents of tags with these names should be
-    # instantiated with some class other than NavigableString.
-    DEFAULT_STRING_CONTAINERS = {}
-    
-    USE_DEFAULT = object()
+    """Turn a textual document into a Beautiful Soup object tree.
+
+    This is an abstract superclass which smooths out the behavior of
+    different parser libraries into a single, unified interface.
+
+    :param multi_valued_attributes: If this is set to None, the
+     TreeBuilder will not turn any values for attributes like
+     'class' into lists. Setting this to a dictionary will
+     customize this behavior; look at DEFAULT_CDATA_LIST_ATTRIBUTES
+     for an example.
+
+     Internally, these are called "CDATA list attributes", but that
+     probably doesn't make sense to an end-user, so the argument name
+     is `multi_valued_attributes`.
+
+    :param preserve_whitespace_tags: A set of tags to treat
+     the way <pre> tags are treated in HTML. Tags in this set
+     are immune from pretty-printing; their contents will always be
+     output as-is.
+
+    :param string_containers: A dictionary mapping tag names to
+    the classes that should be instantiated to contain the textual
+    contents of those tags. The default is to use NavigableString
+    for every tag, no matter what the name. You can override the
+    default by changing DEFAULT_STRING_CONTAINERS.
+
+    :param store_line_numbers: If the parser keeps track of the
+     line numbers and positions of the original markup, that
+     information will, by default, be stored in each corresponding
+     `Tag` object. You can turn this off by passing
+     store_line_numbers=False. If the parser you're using doesn't 
+     keep track of this information, then setting store_line_numbers=True
+     will do nothing.
+    """
 
-    # Most parsers don't keep track of line numbers.
-    TRACKS_LINE_NUMBERS = False
+    USE_DEFAULT: Any = object() #: :meta private:
     
-    def __init__(self, multi_valued_attributes=USE_DEFAULT,
-                 preserve_whitespace_tags=USE_DEFAULT,
-                 store_line_numbers=USE_DEFAULT,
-                 string_containers=USE_DEFAULT,
+    def __init__(self, multi_valued_attributes:Dict[str, Set[str]]=USE_DEFAULT,
+                 preserve_whitespace_tags:Set[str]=USE_DEFAULT,
+                 store_line_numbers:bool=USE_DEFAULT,
+                 string_containers:Dict[str, Type[NavigableString]]=USE_DEFAULT,
+                 empty_element_tags:Set[str]=USE_DEFAULT
     ):
-        """Constructor.
-
-        :param multi_valued_attributes: If this is set to None, the
-         TreeBuilder will not turn any values for attributes like
-         'class' into lists. Setting this to a dictionary will
-         customize this behavior; look at DEFAULT_CDATA_LIST_ATTRIBUTES
-         for an example.
-
-         Internally, these are called "CDATA list attributes", but that
-         probably doesn't make sense to an end-user, so the argument name
-         is `multi_valued_attributes`.
-
-        :param preserve_whitespace_tags: A list of tags to treat
-         the way <pre> tags are treated in HTML. Tags in this list
-         are immune from pretty-printing; their contents will always be
-         output as-is.
-
-        :param string_containers: A dictionary mapping tag names to
-        the classes that should be instantiated to contain the textual
-        contents of those tags. The default is to use NavigableString
-        for every tag, no matter what the name. You can override the
-        default by changing DEFAULT_STRING_CONTAINERS.
-
-        :param store_line_numbers: If the parser keeps track of the
-         line numbers and positions of the original markup, that
-         information will, by default, be stored in each corresponding
-         `Tag` object. You can turn this off by passing
-         store_line_numbers=False. If the parser you're using doesn't 
-         keep track of this information, then setting store_line_numbers=True
-         will do nothing.
-        """
         self.soup = None
         if multi_valued_attributes is self.USE_DEFAULT:
             multi_valued_attributes = self.DEFAULT_CDATA_LIST_ATTRIBUTES
@@ -179,14 +190,55 @@ class TreeBuilder(object):
         if preserve_whitespace_tags is self.USE_DEFAULT:
             preserve_whitespace_tags = self.DEFAULT_PRESERVE_WHITESPACE_TAGS
         self.preserve_whitespace_tags = preserve_whitespace_tags
+        if empty_element_tags is self.USE_DEFAULT:
+            self.empty_element_tags = self.DEFAULT_EMPTY_ELEMENT_TAGS
+        else:
+            self.empty_element_tags = empty_element_tags
         if store_line_numbers == self.USE_DEFAULT:
             store_line_numbers = self.TRACKS_LINE_NUMBERS
         self.store_line_numbers = store_line_numbers 
         if string_containers == self.USE_DEFAULT:
             string_containers = self.DEFAULT_STRING_CONTAINERS
         self.string_containers = string_containers
+
+    NAME:str = "[Unknown tree builder]"
+    ALTERNATE_NAMES: Iterable[str] = []
+    features: Iterable[str] = []
+
+    is_xml: bool = False
+    picklable: bool = False
+
+    soup: Optional[BeautifulSoup] #: :meta private:
+
+    #: A tag will be considered an empty-element
+    #: tag when and only when it has no contents.
+    empty_element_tags: Optional[Set[str]] = None #: :meta private:
+    cdata_list_attributes: Dict[str, Set[str]] #: :meta private:
+    preserve_whitespace_tags: Set[str] #: :meta private:
+    string_containers: Dict[str, Type[NavigableString]] #: :meta private:
+    tracks_line_numbers: bool #: :meta private:
+    
+    #: A value for these tag/attribute combinations is a space- or
+    #: comma-separated list of CDATA, rather than a single CDATA.
+    DEFAULT_CDATA_LIST_ATTRIBUTES : Dict[str, Set[str]] = defaultdict(set)
+
+    #: Whitespace should be preserved inside these tags.
+    DEFAULT_PRESERVE_WHITESPACE_TAGS : Set[str] = set()
+
+    #: The textual contents of tags with these names should be
+    #: instantiated with some class other than NavigableString.
+    DEFAULT_STRING_CONTAINERS: Dict[str, Type[NavigableString]] = {}
+
+    #: By default, tags are treated as empty-element tags if they have
+    #: no contents--that is, using XML rules. HTMLTreeBuilder
+    #: defines a different set of DEFAULT_EMPTY_ELEMENT_TAGS based on the
+    #: HTML 4 and HTML5 standards.
+    DEFAULT_EMPTY_ELEMENT_TAGS: Optional[Set] = None
+    
+    #: Most parsers don't keep track of line numbers.
+    TRACKS_LINE_NUMBERS: bool = False
         
-    def initialize_soup(self, soup):
+    def initialize_soup(self, soup:BeautifulSoup) -> None:
         """The BeautifulSoup object has been initialized and is now
         being associated with the TreeBuilder.
 
@@ -194,7 +246,7 @@ class TreeBuilder(object):
         """
         self.soup = soup
         
-    def reset(self):
+    def reset(self) -> None:
         """Do any work necessary to reset the underlying parser
         for a new document.
 
@@ -202,7 +254,7 @@ class TreeBuilder(object):
         """
         pass
 
-    def can_be_empty_element(self, tag_name):
+    def can_be_empty_element(self, tag_name:str) -> bool:
         """Might a tag with this name be an empty-element tag?
 
         The final markup may or may not actually present this tag as
@@ -225,46 +277,48 @@ class TreeBuilder(object):
             return True
         return tag_name in self.empty_element_tags
     
-    def feed(self, markup):
+    def feed(self, markup:str) -> None:
         """Run some incoming markup through some parsing process,
-        populating the `BeautifulSoup` object in self.soup.
-
-        This method is not implemented in TreeBuilder; it must be
-        implemented in subclasses.
-
-        :return: None.
+        populating the `BeautifulSoup` object in `TreeBuilder.soup`
         """
         raise NotImplementedError()
 
-    def prepare_markup(self, markup, user_specified_encoding=None,
-                       document_declared_encoding=None, exclude_encodings=None):
+    def prepare_markup(
+            self, markup:_RawMarkup,
+            user_specified_encoding:Optional[_Encoding]=None,
+            document_declared_encoding:Optional[_Encoding]=None,
+            exclude_encodings:Optional[_Encodings]=None
+    ) -> Iterable[Tuple[_RawMarkup, Optional[_Encoding], Optional[_Encoding], bool]]:
         """Run any preliminary steps necessary to make incoming markup
         acceptable to the parser.
 
-        :param markup: Some markup -- probably a bytestring.
-        :param user_specified_encoding: The user asked to try this encoding.
+        :param markup: The markup that's about to be parsed.
+        :param user_specified_encoding: The user asked to try this encoding
+           to convert the markup into a Unicode string.
         :param document_declared_encoding: The markup itself claims to be
             in this encoding. NOTE: This argument is not used by the
             calling code and can probably be removed.
-        :param exclude_encodings: The user asked _not_ to try any of
+        :param exclude_encodings: The user asked *not* to try any of
             these encodings.
 
-        :yield: A series of 4-tuples:
-         (markup, encoding, declared encoding,
-          has undergone character replacement)
+        :yield: A series of 4-tuples: (markup, encoding, declared encoding,
+            has undergone character replacement)
 
-         Each 4-tuple represents a strategy for converting the
-         document to Unicode and parsing it. Each strategy will be tried 
-         in turn.
+            Each 4-tuple represents a strategy that the parser can try
+            to convert the document to Unicode and parse it. Each
+            strategy will be tried in turn.
 
          By default, the only strategy is to parse the markup
          as-is. See `LXMLTreeBuilderForXML` and
          `HTMLParserTreeBuilder` for implementations that take into
          account the quirks of particular parsers.
+
+        :meta private:
+
         """
         yield markup, None, None, False
 
-    def test_fragment_to_document(self, fragment):
+    def test_fragment_to_document(self, fragment:str) -> str:
         """Wrap an HTML fragment to make it look like a document.
 
         Different parsers do this differently. For instance, lxml
@@ -273,26 +327,27 @@ class TreeBuilder(object):
         which run HTML fragments through the parser and compare the
         results against other HTML fragments.
 
-        This method should not be used outside of tests.
+        This method should not be used outside of unit tests.
 
-        :param fragment: A string -- fragment of HTML.
-        :return: A string -- a full HTML document.
+        :param fragment: A fragment of HTML.
+        :return: A full HTML document.
+        :meta private:
         """
         return fragment
 
-    def set_up_substitutions(self, tag):
+    def set_up_substitutions(self, tag:Tag) -> bool:
         """Set up any substitutions that will need to be performed on 
         a `Tag` when it's output as a string.
 
         By default, this does nothing. See `HTMLTreeBuilder` for a
         case where this is used.
 
-        :param tag: A `Tag`
         :return: Whether or not a substitution was performed.
+        :meta private:
         """
         return False
 
-    def _replace_cdata_list_attribute_values(self, tag_name, attrs):
+    def _replace_cdata_list_attribute_values(self, tag_name:str, attrs:_AttributeValues):
         """When an attribute value is associated with a tag that can
         have multiple values for that attribute, convert the string
         value to a list of strings.
@@ -308,10 +363,11 @@ class TreeBuilder(object):
         if not attrs:
             return attrs
         if self.cdata_list_attributes:
-            universal = self.cdata_list_attributes.get('*', [])
+            universal: Set[str] = self.cdata_list_attributes.get('*', set())
             tag_specific = self.cdata_list_attributes.get(
                 tag_name.lower(), None)
             for attr in list(attrs.keys()):
+                values: _AttributeValue
                 if attr in universal or (tag_specific and attr in tag_specific):
                     # We have a "class"-type attribute whose string
                     # value is a whitespace-separated list of
@@ -337,7 +393,15 @@ class SAXTreeBuilder(TreeBuilder):
     how a simple TreeBuilder would work.
     """
 
-    def feed(self, markup):
+    def __init__(self, *args, **kwargs):
+        warnings.warn(
+            f"The SAXTreeBuilder class was deprecated in 4.13.0. It is completely untested and probably doesn't work; use at your own risk.",
+                DeprecationWarning,
+                stacklevel=2
+            )
+        super(SAXTreeBuilder, self).__init__(*args, **kwargs)
+    
+    def feed(self, markup:_RawMarkup):
         raise NotImplementedError()
 
     def close(self):
@@ -381,12 +445,13 @@ class SAXTreeBuilder(TreeBuilder):
 
 
 class HTMLTreeBuilder(TreeBuilder):
-    """This TreeBuilder knows facts about HTML.
-
-    Such as which tags are empty-element tags.
+    """This TreeBuilder knows facts about HTML, such as which tags are treated
+    specially by the HTML standard.
     """
 
-    empty_element_tags = set([
+    #: Some HTML tags are defined as having no contents. Beautiful Soup
+    #: treats these specially.
+    DEFAULT_EMPTY_ELEMENT_TAGS: Set[str] = set([
         # These are from HTML5.
         'area', 'base', 'br', 'col', 'embed', 'hr', 'img', 'input', 'keygen', 'link', 'menuitem', 'meta', 'param', 'source', 'track', 'wbr',
         
@@ -394,29 +459,29 @@ class HTMLTreeBuilder(TreeBuilder):
         'basefont', 'bgsound', 'command', 'frame', 'image', 'isindex', 'nextid', 'spacer'
     ])
 
-    # The HTML standard defines these as block-level elements. Beautiful
-    # Soup does not treat these elements differently from other elements,
-    # but it may do so eventually, and this information is available if
-    # you need to use it.
-    block_elements = set(["address", "article", "aside", "blockquote", "canvas", "dd", "div", "dl", "dt", "fieldset", "figcaption", "figure", "footer", "form", "h1", "h2", "h3", "h4", "h5", "h6", "header", "hr", "li", "main", "nav", "noscript", "ol", "output", "p", "pre", "section", "table", "tfoot", "ul", "video"])
-
-    # These HTML tags need special treatment so they can be
-    # represented by a string class other than NavigableString.
-    #
-    # For some of these tags, it's because the HTML standard defines
-    # an unusual content model for them. I made this list by going
-    # through the HTML spec
-    # (https://html.spec.whatwg.org/#metadata-content) and looking for
-    # "metadata content" elements that can contain strings.
-    #
-    # The Ruby tags (<rt> and <rp>) are here despite being normal
-    # "phrasing content" tags, because the content they contain is
-    # qualitatively different from other text in the document, and it
-    # can be useful to be able to distinguish it.
-    #
-    # TODO: Arguably <noscript> could go here but it seems
-    # qualitatively different from the other tags.
-    DEFAULT_STRING_CONTAINERS = {
+    #: The HTML standard defines these tags as block-level elements. Beautiful
+    #: Soup does not treat these elements differently from other elements,
+    #: but it may do so eventually, and this information is available if
+    #: you need to use it.
+    DEFAULT_BLOCK_ELEMENTS: Set[str] = set(["address", "article", "aside", "blockquote", "canvas", "dd", "div", "dl", "dt", "fieldset", "figcaption", "figure", "footer", "form", "h1", "h2", "h3", "h4", "h5", "h6", "header", "hr", "li", "main", "nav", "noscript", "ol", "output", "p", "pre", "section", "table", "tfoot", "ul", "video"])
+
+    #: These HTML tags need special treatment so they can be
+    #: represented by a string class other than NavigableString.
+    #:
+    #: For some of these tags, it's because the HTML standard defines
+    #: an unusual content model for them. I made this list by going
+    #: through the HTML spec
+    #: (https://html.spec.whatwg.org/#metadata-content) and looking for
+    #: "metadata content" elements that can contain strings.
+    #:
+    #: The Ruby tags (<rt> and <rp>) are here despite being normal
+    #: "phrasing content" tags, because the content they contain is
+    #: qualitatively different from other text in the document, and it
+    #: can be useful to be able to distinguish it.
+    #:
+    #: TODO: Arguably <noscript> could go here but it seems
+    #: qualitatively different from the other tags.
+    DEFAULT_STRING_CONTAINERS: Dict[str, Type[NavigableString]] = {
         'rt' : RubyTextString,
         'rp' : RubyParenthesisString,
         'style': Stylesheet,
@@ -424,33 +489,35 @@ class HTMLTreeBuilder(TreeBuilder):
         'template': TemplateString,
     }    
     
-    # The HTML standard defines these attributes as containing a
-    # space-separated list of values, not a single value. That is,
-    # class="foo bar" means that the 'class' attribute has two values,
-    # 'foo' and 'bar', not the single value 'foo bar'.  When we
-    # encounter one of these attributes, we will parse its value into
-    # a list of values if possible. Upon output, the list will be
-    # converted back into a string.
-    DEFAULT_CDATA_LIST_ATTRIBUTES = {
-        "*" : ['class', 'accesskey', 'dropzone'],
-        "a" : ['rel', 'rev'],
-        "link" :  ['rel', 'rev'],
-        "td" : ["headers"],
-        "th" : ["headers"],
-        "td" : ["headers"],
-        "form" : ["accept-charset"],
-        "object" : ["archive"],
+    #: The HTML standard defines these attributes as containing a
+    #: space-separated list of values, not a single value. That is,
+    #: class="foo bar" means that the 'class' attribute has two values,
+    #: 'foo' and 'bar', not the single value 'foo bar'.  When we
+    #: encounter one of these attributes, we will parse its value into
+    #: a list of values if possible. Upon output, the list will be
+    #: converted back into a string.
+    DEFAULT_CDATA_LIST_ATTRIBUTES: Dict[str, Set[str]] = {
+        "*" : {'class', 'accesskey', 'dropzone'},
+        "a" : {'rel', 'rev'},
+        "link" :  {'rel', 'rev'},
+        "td" : {"headers"},
+        "th" : {"headers"},
+        "td" : {"headers"},
+        "form" : {"accept-charset"},
+        "object" : {"archive"},
 
         # These are HTML5 specific, as are *.accesskey and *.dropzone above.
-        "area" : ["rel"],
-        "icon" : ["sizes"],
-        "iframe" : ["sandbox"],
-        "output" : ["for"],
+        "area" : {"rel"},
+        "icon" : {"sizes"},
+        "iframe" : {"sandbox"},
+        "output" : {"for"},
         }
 
-    DEFAULT_PRESERVE_WHITESPACE_TAGS = set(['pre', 'textarea'])
+    #: By default, whitespace inside these HTML tags will be
+    #: preserved rather than being collapsed.
+    DEFAULT_PRESERVE_WHITESPACE_TAGS: set[str] = set(['pre', 'textarea'])
 
-    def set_up_substitutions(self, tag):
+    def set_up_substitutions(self, tag:Tag) -> bool:
         """Replace the declared encoding in a <meta> tag with a placeholder,
         to be substituted when the tag is output to a string.
 
@@ -458,17 +525,26 @@ class HTMLTreeBuilder(TreeBuilder):
         encoding, but exit in a different encoding, and the <meta> tag
         needs to be changed to reflect this.
 
-        :param tag: A `Tag`
         :return: Whether or not a substitution was performed.
+
+        :meta private:
         """
         # We are only interested in <meta> tags
         if tag.name != 'meta':
             return False
 
-        http_equiv = tag.get('http-equiv')
-        content = tag.get('content')
-        charset = tag.get('charset')
-
+        # TODO: This cast will fail in the (very unlikely) scenario
+        # that the programmer who instantiates the TreeBuilder
+        # specifies meta['content'] or meta['charset'] as
+        # cdata_list_attributes.
+        content:Optional[str] = cast(Optional[str], tag.get('content'))
+        charset:Optional[str] = cast(Optional[str], tag.get('charset'))
+
+        # But we can accommodate meta['http-equiv'] being made a
+        # cdata_list_attribute (again, very unlikely) without much
+        # trouble.
+        http_equiv:List[str] = tag.get_attribute_list('http-equiv')
+        
         # We are interested in <meta> tags that say what encoding the
         # document was originally in. This means HTML 5-style <meta>
         # tags that provide the "charset" attribute. It also means
@@ -478,20 +554,22 @@ class HTMLTreeBuilder(TreeBuilder):
         # In both cases we will replace the value of the appropriate
         # attribute with a standin object that can take on any
         # encoding.
-        meta_encoding = None
+        substituted = False
         if charset is not None:
             # HTML 5 style:
             # <meta charset="utf8">
             meta_encoding = charset
             tag['charset'] = CharsetMetaAttributeValue(charset)
+            substituted = True
 
-        elif (content is not None and http_equiv is not None
-              and http_equiv.lower() == 'content-type'):
+        elif (content is not None and
+              any(x.lower() == 'content-type' for x in http_equiv)):
             # HTML 4 style:
             # <meta http-equiv="content-type" content="text/html; charset=utf8">
             tag['content'] = ContentMetaAttributeValue(content)
+            substituted = True
 
-        return (meta_encoding is not None)
+        return substituted
 
 class DetectsXMLParsedAsHTML(object):
     """A mixin class for any class (a TreeBuilder, or some class used by a
@@ -502,19 +580,29 @@ class DetectsXMLParsedAsHTML(object):
     This requires being able to observe an incoming processing
     instruction that might be an XML declaration, and also able to
     observe tags as they're opened. If you can't do that for a given
-    TreeBuilder, there's a less reliable implementation based on
+    `TreeBuilder`, there's a less reliable implementation based on
     examining the raw markup.
     """
 
-    # Regular expression for seeing if markup has an <html> tag.
-    LOOKS_LIKE_HTML = re.compile("<[^ +]html", re.I)
-    LOOKS_LIKE_HTML_B = re.compile(b"<[^ +]html", re.I)
+    #: Regular expression for seeing if string markup has an <html> tag.
+    LOOKS_LIKE_HTML:Pattern[str] = re.compile("<[^ +]html", re.I)
 
-    XML_PREFIX = '<?xml'
-    XML_PREFIX_B = b'<?xml'
+    #: Regular expression for seeing if byte markup has an <html> tag.
+    LOOKS_LIKE_HTML_B:Pattern[bytes] = re.compile(b"<[^ +]html", re.I)
+
+    #: The start of an XML document string.
+    XML_PREFIX:str = '<?xml'
+
+    #: The start of an XML document bytestring.
+    XML_PREFIX_B:bytes = b'<?xml'
+
+    # This is typed as str, not `ProcessingInstruction`, because this
+    # check may be run before any Beautiful Soup objects are created.
+    _first_processing_instruction: Optional[str]
+    _root_tag: Optional[Tag]
     
     @classmethod
-    def warn_if_markup_looks_like_xml(cls, markup):
+    def warn_if_markup_looks_like_xml(cls, markup:Optional[_RawMarkup]) -> bool:
         """Perform a check on some markup to see if it looks like XML
         that's not XHTML. If so, issue a warning.
 
@@ -524,34 +612,40 @@ class DetectsXMLParsedAsHTML(object):
         :return: True if the markup looks like non-XHTML XML, False
         otherwise.
         """
+        if markup is None:
+            return False
+        markup = markup[:500]
         if isinstance(markup, bytes):
-            prefix = cls.XML_PREFIX_B
-            looks_like_html = cls.LOOKS_LIKE_HTML_B
+            markup_b = cast(bytes, markup)
+            looks_like_xml = (
+                markup_b.startswith(cls.XML_PREFIX_B)
+                and not cls.LOOKS_LIKE_HTML_B.search(markup)
+            )
         else:
-            prefix = cls.XML_PREFIX
-            looks_like_html = cls.LOOKS_LIKE_HTML
-        
-        if (markup is not None
-            and markup.startswith(prefix)
-            and not looks_like_html.search(markup[:500])
-        ):
+            markup_s = cast(str, markup)
+            looks_like_xml = (
+                markup_s.startswith(cls.XML_PREFIX)
+                and not cls.LOOKS_LIKE_HTML.search(markup)
+            )
+
+        if looks_like_xml:
             cls._warn()
             return True
-        return False
-
+        return False        
+    
     @classmethod
-    def _warn(cls):
+    def _warn(cls) -> None:
         """Issue a warning about XML being parsed as HTML."""
         warnings.warn(
             XMLParsedAsHTMLWarning.MESSAGE, XMLParsedAsHTMLWarning
         )
         
-    def _initialize_xml_detector(self):
+    def _initialize_xml_detector(self) -> None:
         """Call this method before parsing a document."""
         self._first_processing_instruction = None
         self._root_tag = None
        
-    def _document_might_be_xml(self, processing_instruction):
+    def _document_might_be_xml(self, processing_instruction:str):
         """Call this method when encountering an XML declaration, or a
         "processing instruction" that might be an XML declaration.
         """
@@ -586,7 +680,7 @@ class DetectsXMLParsedAsHTML(object):
             self._warn()
 
     
-def register_treebuilders_from(module):
+def register_treebuilders_from(module:ModuleType) -> None:
     """Copy TreeBuilders from the given module into this module."""
     this_module = sys.modules[__name__]
     for name in module.__all__:
@@ -602,7 +696,7 @@ class ParserRejectedMarkup(Exception):
     """An Exception to be raised when the underlying parser simply
     refuses to parse the given markup.
     """
-    def __init__(self, message_or_exception):
+    def __init__(self, message_or_exception:Union[str,Exception]):
         """Explain why the parser rejected the given markup, either
         with a textual explanation or another exception.
         """
diff --git a/bs4/builder/_html5lib.py b/bs4/builder/_html5lib.py
index dac2173..560a036 100644
--- a/bs4/builder/_html5lib.py
+++ b/bs4/builder/_html5lib.py
@@ -5,6 +5,20 @@ __all__ = [
     'HTML5TreeBuilder',
     ]
 
+from typing import (
+    Iterable,
+    List,
+    Optional,
+    TYPE_CHECKING,
+    Tuple,
+    Union,
+)
+from bs4._typing import (
+    _Encoding,
+    _Encodings,
+    _RawMarkup,
+)
+
 import warnings
 import re
 from bs4.builder import (
@@ -30,50 +44,54 @@ from bs4.element import (
     Tag,
     )
 
-try:
-    # Pre-0.99999999
-    from html5lib.treebuilders import _base as treebuilder_base
-    new_html5lib = False
-except ImportError as e:
-    # 0.99999999 and up
-    from html5lib.treebuilders import base as treebuilder_base
-    new_html5lib = True
+from html5lib.treebuilders import base as treebuilder_base
+
 
 class HTML5TreeBuilder(HTMLTreeBuilder):
-    """Use html5lib to build a tree.
+    """Use `html5lib <https://github.com/html5lib/html5lib-python>`_ to
+    build a tree.
 
-    Note that this TreeBuilder does not support some features common
-    to HTML TreeBuilders. Some of these features could theoretically
+    Note that `HTML5TreeBuilder` does not support some common HTML
+    `TreeBuilder` features. Some of these features could theoretically
     be implemented, but at the very least it's quite difficult,
     because html5lib moves the parse tree around as it's being built.
 
-    * This TreeBuilder doesn't use different subclasses of NavigableString
-      based on the name of the tag in which the string was found.
+    Specifically:
 
-    * You can't use a SoupStrainer to parse only part of a document.
+    * This `TreeBuilder` doesn't use different subclasses of
+      `NavigableString` (e.g. `Script`) based on the name of the tag
+      in which the string was found.
+    * You can't use a `SoupStrainer` to parse only part of a document.
     """
 
-    NAME = "html5lib"
+    NAME:str = "html5lib"
 
-    features = [NAME, PERMISSIVE, HTML_5, HTML]
+    features:Iterable[str] = [NAME, PERMISSIVE, HTML_5, HTML]
 
-    # html5lib can tell us which line number and position in the
-    # original file is the source of an element.
-    TRACKS_LINE_NUMBERS = True
+    #: html5lib can tell us which line number and position in the
+    #: original file is the source of an element.
+    TRACKS_LINE_NUMBERS:bool = True
     
-    def prepare_markup(self, markup, user_specified_encoding,
-                       document_declared_encoding=None, exclude_encodings=None):
+    def prepare_markup(self, markup:_RawMarkup,
+                       user_specified_encoding:Optional[_Encoding]=None,
+                       document_declared_encoding:Optional[_Encoding]=None,
+                       exclude_encodings:Optional[_Encodings]=None
+        ) -> Iterable[Tuple[_RawMarkup, Optional[_Encoding], Optional[_Encoding], bool]]:
         # Store the user-specified encoding for use later on.
         self.user_specified_encoding = user_specified_encoding
 
         # document_declared_encoding and exclude_encodings aren't used
         # ATM because the html5lib TreeBuilder doesn't use
         # UnicodeDammit.
-        if exclude_encodings:
-            warnings.warn(
-                "You provided a value for exclude_encoding, but the html5lib tree builder doesn't support exclude_encoding.",
-                stacklevel=3
-            )
+        for variable, name in (
+            (document_declared_encoding, 'document_declared_encoding'),
+            (exclude_encodings, 'exclude_encodings'),
+        ):
+            if variable:
+                warnings.warn(
+                    f"You provided a value for {name}, but the html5lib tree builder doesn't support {name}.",
+                    stacklevel=3
+                )
 
         # html5lib only parses HTML, so if it's given XML that's worth
         # noting.
@@ -83,6 +101,9 @@ class HTML5TreeBuilder(HTMLTreeBuilder):
 
     # These methods are defined by Beautiful Soup.
     def feed(self, markup):
+        """Run some incoming markup through some parsing process,
+        populating the `BeautifulSoup` object in `HTML5TreeBuilder.soup`.
+        """
         if self.soup.parse_only is not None:
             warnings.warn(
                 "You provided a value for parse_only, but the html5lib tree builder doesn't support parse_only. The entire document will be parsed.",
@@ -92,10 +113,7 @@ class HTML5TreeBuilder(HTMLTreeBuilder):
         self.underlying_builder.parser = parser
         extra_kwargs = dict()
         if not isinstance(markup, str):
-            if new_html5lib:
-                extra_kwargs['override_encoding'] = self.user_specified_encoding
-            else:
-                extra_kwargs['encoding'] = self.user_specified_encoding
+            extra_kwargs['override_encoding'] = self.user_specified_encoding
         doc = parser.parse(markup, **extra_kwargs)
         
         # Set the character encoding detected by the tokenizer.
@@ -105,15 +123,18 @@ class HTML5TreeBuilder(HTMLTreeBuilder):
             doc.original_encoding = None
         else:
             original_encoding = parser.tokenizer.stream.charEncoding[0]
-            if not isinstance(original_encoding, str):
-                # In 0.99999999 and up, the encoding is an html5lib
-                # Encoding object. We want to use a string for compatibility
-                # with other tree builders.
-                original_encoding = original_encoding.name
+            # The encoding is an html5lib Encoding object. We want to
+            # use a string for compatibility with other tree builders.
+            original_encoding = original_encoding.name
             doc.original_encoding = original_encoding
         self.underlying_builder.parser = None
-            
+
     def create_treebuilder(self, namespaceHTMLElements):
+        """Called by html5lib to instantiate the kind of class it
+        calls a 'TreeBuilder'.
+        
+        :meta private:
+        """
         self.underlying_builder = TreeBuilderForHtml5lib(
             namespaceHTMLElements, self.soup,
             store_line_numbers=self.store_line_numbers
diff --git a/bs4/builder/_htmlparser.py b/bs4/builder/_htmlparser.py
index 3cc187f..291f6c6 100644
--- a/bs4/builder/_htmlparser.py
+++ b/bs4/builder/_htmlparser.py
@@ -1,4 +1,5 @@
 # encoding: utf-8
+from __future__ import annotations
 """Use the HTMLParser library to parse HTML files that aren't too bad."""
 
 # Use of this source code is governed by the MIT license.
@@ -11,6 +12,19 @@ __all__ = [
 from html.parser import HTMLParser
 
 import sys
+from typing import (
+    Any,
+    Callable,
+    cast,
+    Dict,
+    Iterable,
+    List,
+    Optional,
+    TYPE_CHECKING,
+    Tuple,
+    Type,
+    Union,
+)
 import warnings
 
 from bs4.element import (
@@ -30,21 +44,25 @@ from bs4.builder import (
     STRICT,
     )
 
-
+from bs4.element import Tag
+if TYPE_CHECKING:
+    from bs4 import BeautifulSoup
+    from bs4.element import NavigableString
+    from bs4._typing import (
+        _AttributeValues,
+        _Encoding,
+        _Encodings,
+        _RawMarkup,
+    )
+    
 HTMLPARSER = 'html.parser'
 
+_DuplicateAttributeHandler = Callable[[Dict[str, str], str, str], None]
+
 class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
     """A subclass of the Python standard library's HTMLParser class, which
     listens for HTMLParser events and translates them into calls
     to Beautiful Soup's tree construction API.
-    """
-
-    # Strategies for handling duplicate attributes
-    IGNORE = 'ignore'
-    REPLACE = 'replace'
-    
-    def __init__(self, *args, **kwargs):
-        """Constructor.
 
         :param on_duplicate_attribute: A strategy for what to do if a
             tag includes the same attribute more than once. Accepted
@@ -53,8 +71,10 @@ class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
             encountered), or a callable. A callable must take three
             arguments: the dictionary of attributes already processed,
             the name of the duplicate attribute, and the most recent value
-            encountered.           
-        """
+            encountered.
+    """
+    def __init__(self, soup:BeautifulSoup, *args, **kwargs):
+        self.soup = soup
         self.on_duplicate_attribute = kwargs.pop(
             'on_duplicate_attribute', self.REPLACE
         )
@@ -70,8 +90,20 @@ class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
         self.already_closed_empty_element = []
 
         self._initialize_xml_detector()
+    
+    #: Constant to handle duplicate attributes by replacing earlier values
+    #: with later ones.
+    IGNORE:str = 'ignore'
+
+    #: Constant to handle duplicate attributes by ignoring later values
+    #: and keeping the earlier ones.    
+    REPLACE:str = 'replace'
 
-    def error(self, message):
+    on_duplicate_attribute:Union[str, _DuplicateAttributeHandler]
+    already_closed_empty_element: List[str]
+    soup: BeautifulSoup
+    
+    def error(self, message:str) -> None:
         # NOTE: This method is required so long as Python 3.9 is
         # supported. The corresponding code is removed from HTMLParser
         # in 3.5, but not removed from ParserBase until 3.10.
@@ -87,32 +119,33 @@ class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
         # catch this error and wrap it in a ParserRejectedMarkup.)
         raise ParserRejectedMarkup(message)
 
-    def handle_startendtag(self, name, attrs):
+    def handle_startendtag(
+            self, name:str, attrs:List[Tuple[str, Optional[str]]]
+    ) -> None:
         """Handle an incoming empty-element tag.
 
-        This is only called when the markup looks like <tag/>.
-
-        :param name: Name of the tag.
-        :param attrs: Dictionary of the tag's attributes.
+        html.parser only calls this method when the markup looks like
+        <tag/>.
         """
-        # is_startend() tells handle_starttag not to close the tag
+        # `handle_empty_element` tells handle_starttag not to close the tag
         # just because its name matches a known empty-element tag. We
-        # know that this is an empty-element tag and we want to call
+        # know that this is an empty-element tag, and we want to call
         # handle_endtag ourselves.
-        tag = self.handle_starttag(name, attrs, handle_empty_element=False)
+        self.handle_starttag(name, attrs, handle_empty_element=False)
         self.handle_endtag(name)
         
-    def handle_starttag(self, name, attrs, handle_empty_element=True):
+    def handle_starttag(
+            self, name:str, attrs:List[Tuple[str, Optional[str]]],
+            handle_empty_element:bool=True
+    ) -> None:
         """Handle an opening tag, e.g. '<tag>'
 
-        :param name: Name of the tag.
-        :param attrs: Dictionary of the tag's attributes.
         :param handle_empty_element: True if this tag is known to be
             an empty-element tag (i.e. there is not expected to be any
             closing tag).
         """
-        # XXX namespace
-        attr_dict = {}
+        # TODO: handle namespaces here?
+        attr_dict: Dict[str, str] = {}
         for key, value in attrs:
             # Change None attribute values to the empty string
             # for consistency with the other tree builders.
@@ -128,6 +161,7 @@ class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
                 elif on_dupe in (None, self.REPLACE):
                     attr_dict[key] = value
                 else:
+                    on_dupe = cast(_DuplicateAttributeHandler, on_dupe)
                     on_dupe(attr_dict, key, value)
             else:
                 attr_dict[key] = value
@@ -157,7 +191,7 @@ class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
         if self._root_tag is None:
             self._root_tag_encountered(name)
             
-    def handle_endtag(self, name, check_already_closed=True):
+    def handle_endtag(self, name:str, check_already_closed:bool=True) -> None:
         """Handle a closing tag, e.g. '</tag>'
         
         :param name: A tag name.
@@ -175,11 +209,11 @@ class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
         else:
             self.soup.handle_endtag(name)
             
-    def handle_data(self, data):
+    def handle_data(self, data:str) -> None:
         """Handle some textual data that shows up between tags."""
         self.soup.handle_data(data)
 
-    def handle_charref(self, name):
+    def handle_charref(self, name:str) -> None:
         """Handle a numeric character reference by converting it to the
         corresponding Unicode character and treating it as textual
         data.
@@ -219,7 +253,7 @@ class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
         data = data or "\N{REPLACEMENT CHARACTER}"
         self.handle_data(data)
 
-    def handle_entityref(self, name):
+    def handle_entityref(self, name:str) -> None:
         """Handle a named entity reference by converting it to the
         corresponding Unicode character(s) and treating it as textual
         data.
@@ -238,7 +272,7 @@ class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
             data = "&%s" % name
         self.handle_data(data)
 
-    def handle_comment(self, data):
+    def handle_comment(self, data:str) -> None:
         """Handle an HTML comment.
 
         :param data: The text of the comment.
@@ -247,7 +281,7 @@ class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
         self.soup.handle_data(data)
         self.soup.endData(Comment)
 
-    def handle_decl(self, data):
+    def handle_decl(self, data:str) -> None:
         """Handle a DOCTYPE declaration.
 
         :param data: The text of the declaration.
@@ -257,11 +291,12 @@ class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
         self.soup.handle_data(data)
         self.soup.endData(Doctype)
 
-    def unknown_decl(self, data):
+    def unknown_decl(self, data:str) -> None:
         """Handle a declaration of unknown type -- probably a CDATA block.
 
         :param data: The text of the declaration.
         """
+        cls: Type[NavigableString]
         if data.upper().startswith('CDATA['):
             cls = CData
             data = data[len('CDATA['):]
@@ -271,7 +306,7 @@ class BeautifulSoupHTMLParser(HTMLParser, DetectsXMLParsedAsHTML):
         self.soup.handle_data(data)
         self.soup.endData(cls)
 
-    def handle_pi(self, data):
+    def handle_pi(self, data:str) -> None:
         """Handle a processing instruction.
 
         :param data: The text of the instruction.
@@ -286,16 +321,17 @@ class HTMLParserTreeBuilder(HTMLTreeBuilder):
     """A Beautiful soup `TreeBuilder` that uses the `HTMLParser` parser,
     found in the Python standard library.
     """
-    is_xml = False
-    picklable = True
-    NAME = HTMLPARSER
-    features = [NAME, HTML, STRICT]
+    is_xml:bool = False
+    picklable:bool = True
+    NAME:str = HTMLPARSER
+    features: Iterable[str] = [NAME, HTML, STRICT]
 
-    # The html.parser knows which line number and position in the
-    # original file is the source of an element.
-    TRACKS_LINE_NUMBERS = True
+    #: The html.parser knows which line number and position in the
+    #: original file is the source of an element.
+    TRACKS_LINE_NUMBERS:bool = True
 
-    def __init__(self, parser_args=None, parser_kwargs=None, **kwargs):
+    def __init__(self, parser_args:Optional[Iterable[Any]]=None,
+                 parser_kwargs:Optional[Dict[str, Any]]=None, **kwargs:Any):
         """Constructor.
 
         :param parser_args: Positional arguments to pass into 
@@ -320,9 +356,12 @@ class HTMLParserTreeBuilder(HTMLTreeBuilder):
         parser_kwargs['convert_charrefs'] = False
         self.parser_args = (parser_args, parser_kwargs)
         
-    def prepare_markup(self, markup, user_specified_encoding=None,
-                       document_declared_encoding=None, exclude_encodings=None):
-
+    def prepare_markup(
+            self, markup:_RawMarkup,
+            user_specified_encoding:Optional[_Encoding]=None,
+            document_declared_encoding:Optional[_Encoding]=None,
+            exclude_encodings:Optional[_Encodings]=None
+    ) -> Iterable[Tuple[str, Optional[_Encoding], Optional[_Encoding], bool]]:
         """Run any preliminary steps necessary to make incoming markup
         acceptable to the parser.
 
@@ -333,13 +372,12 @@ class HTMLParserTreeBuilder(HTMLTreeBuilder):
         :param exclude_encodings: The user asked _not_ to try any of
             these encodings.
 
-        :yield: A series of 4-tuples:
-         (markup, encoding, declared encoding,
-          has undergone character replacement)
+        :yield: A series of 4-tuples: (markup, encoding, declared encoding,
+             has undergone character replacement)
 
-         Each 4-tuple represents a strategy for converting the
-         document to Unicode and parsing it. Each strategy will be tried 
-         in turn.
+            Each 4-tuple represents a strategy for parsing the document.
+            This TreeBuilder uses Unicode, Dammit to convert the markup
+            into Unicode, so the `markup` element will always be a string.
         """
         if isinstance(markup, str):
             # Parse Unicode as-is.
@@ -348,14 +386,19 @@ class HTMLParserTreeBuilder(HTMLTreeBuilder):
 
         # Ask UnicodeDammit to sniff the most likely encoding.
 
-        # This was provided by the end-user; treat it as a known
-        # definite encoding per the algorithm laid out in the HTML5
-        # spec.  (See the EncodingDetector class for details.)
-        known_definite_encodings = [user_specified_encoding]
+        known_definite_encodings: List[_Encoding] = []
+        if user_specified_encoding:
+            # This was provided by the end-user; treat it as a known
+            # definite encoding per the algorithm laid out in the
+            # HTML5 spec. (See the EncodingDetector class for
+            # details.)
+            known_definite_encodings.append(user_specified_encoding)
 
-        # This was found in the document; treat it as a slightly lower-priority
-        # user encoding.
-        user_encodings = [document_declared_encoding]
+        user_encodings: List[_Encoding] = []
+        if document_declared_encoding:
+            # This was found in the document; treat it as a slightly
+            # lower-priority user encoding.
+            user_encodings.append(document_declared_encoding)
 
         try_encodings = [user_specified_encoding, document_declared_encoding]
         dammit = UnicodeDammit(
@@ -365,17 +408,27 @@ class HTMLParserTreeBuilder(HTMLTreeBuilder):
             is_html=True,
             exclude_encodings=exclude_encodings
         )
-        yield (dammit.markup, dammit.original_encoding,
-               dammit.declared_html_encoding,
-               dammit.contains_replacement_characters)
 
-    def feed(self, markup):
-        """Run some incoming markup through some parsing process,
-        populating the `BeautifulSoup` object in self.soup.
-        """
+        if dammit.unicode_markup is None:
+            # In every case I've seen, Unicode, Dammit is able to
+            # convert the markup into Unicode, even if it needs to use
+            # REPLACEMENT CHARACTER. But there is a code path that
+            # could result in unicode_markup being None, and
+            # HTMLParser can only parse Unicode, so here we handle
+            # that code path.
+            raise ParserRejectedMarkup("Could not convert input to Unicode, and html.parser will not accept bytestrings.")
+        else:
+            yield (dammit.unicode_markup, dammit.original_encoding,
+                   dammit.declared_html_encoding,
+                   dammit.contains_replacement_characters)
+
+    def feed(self, markup:str):
         args, kwargs = self.parser_args
-        parser = BeautifulSoupHTMLParser(*args, **kwargs)
-        parser.soup = self.soup
+        # We know BeautifulSoup calls TreeBuilder.initialize_soup
+        # before calling feed(), so we can assume self.soup
+        # is set.
+        assert self.soup is not None
+        parser = BeautifulSoupHTMLParser(self.soup, *args, **kwargs)
         try:
             parser.feed(markup)
             parser.close()
@@ -385,3 +438,4 @@ class HTMLParserTreeBuilder(HTMLTreeBuilder):
             # when there's an error in the doctype declaration.
             raise ParserRejectedMarkup(e)
         parser.already_closed_empty_element = []
+
diff --git a/bs4/builder/_lxml.py b/bs4/builder/_lxml.py
index 971c81e..44a477f 100644
--- a/bs4/builder/_lxml.py
+++ b/bs4/builder/_lxml.py
@@ -1,3 +1,6 @@
+# encoding: utf-8
+from __future__ import annotations
+
 # Use of this source code is governed by the MIT license.
 __license__ = "MIT"
 
@@ -6,14 +9,26 @@ __all__ = [
     'LXMLTreeBuilder',
     ]
 
-try:
-    from collections.abc import Callable # Python 3.6
-except ImportError as e:
-    from collections import Callable
+from collections.abc import Callable
+
+from typing import (
+    Any,
+    Dict,
+    IO,
+    Iterable,
+    List,
+    Optional,
+    Set,
+    Tuple,
+    Type,
+    TYPE_CHECKING,
+    Union,
+)
 
 from io import BytesIO
 from io import StringIO
 from lxml import etree
+from bs4.dammit import (_Encoding)
 from bs4.element import (
     Comment,
     Doctype,
@@ -31,33 +46,54 @@ from bs4.builder import (
     TreeBuilder,
     XML)
 from bs4.dammit import EncodingDetector
-
-LXML = 'lxml'
+if TYPE_CHECKING:
+    from bs4._typing import (
+        _Encoding,
+        _Encodings,
+        _NamespacePrefix,
+        _NamespaceURL,
+        _NamespaceMapping,
+        _InvertedNamespaceMapping,
+        _RawMarkup,
+    )
+    from bs4 import BeautifulSoup
+
+LXML:str = 'lxml'
 
 def _invert(d):
     "Invert a dictionary."
     return dict((v,k) for k, v in list(d.items()))
 
 class LXMLTreeBuilderForXML(TreeBuilder):
-    DEFAULT_PARSER_CLASS = etree.XMLParser
-
-    is_xml = True
-    processing_instruction_class = XMLProcessingInstruction
 
-    NAME = "lxml-xml"
-    ALTERNATE_NAMES = ["xml"]
+    DEFAULT_PARSER_CLASS:Type[Any] = etree.XMLParser
+    
+    is_xml:bool = True
+        
+    processing_instruction_class:Type[ProcessingInstruction]
+    
+    NAME:str = "lxml-xml"
+    ALTERNATE_NAMES: Iterable[str] = ["xml"]
 
     # Well, it's permissive by XML parser standards.
-    features = [NAME, LXML, XML, FAST, PERMISSIVE]
+    features: Iterable[str] = [NAME, LXML, XML, FAST, PERMISSIVE]
 
-    CHUNK_SIZE = 512
+    CHUNK_SIZE:int = 512
 
     # This namespace mapping is specified in the XML Namespace
     # standard.
-    DEFAULT_NSMAPS = dict(xml='http://www.w3.org/XML/1998/namespace')
+    DEFAULT_NSMAPS: _NamespaceMapping = dict(
+        xml='http://www.w3.org/XML/1998/namespace'
+    )
 
-    DEFAULT_NSMAPS_INVERTED = _invert(DEFAULT_NSMAPS)
+    DEFAULT_NSMAPS_INVERTED:_InvertedNamespaceMapping = _invert(
+        DEFAULT_NSMAPS
+    )
 
+    nsmaps: List[Optional[_InvertedNamespaceMapping]]
+    empty_element_tags: Set[str]
+    parser: Any
+    
     # NOTE: If we parsed Element objects and looked at .sourceline,
     # we'd be able to see the line numbers from the original document.
     # But instead we build an XMLParser or HTMLParser object to serve
@@ -65,16 +101,18 @@ class LXMLTreeBuilderForXML(TreeBuilder):
     # line numbers.
     # See: https://bugs.launchpad.net/lxml/+bug/1846906
     
-    def initialize_soup(self, soup):
+    def initialize_soup(self, soup:BeautifulSoup) -> None:
         """Let the BeautifulSoup object know about the standard namespace
         mapping.
 
         :param soup: A `BeautifulSoup`.
         """
+        # Beyond this point, self.soup is set, so we can assume (and
+        # assert) it's not None whenever necessary.
         super(LXMLTreeBuilderForXML, self).initialize_soup(soup)
         self._register_namespaces(self.DEFAULT_NSMAPS)
 
-    def _register_namespaces(self, mapping):
+    def _register_namespaces(self, mapping:Dict[str, str]) -> None:
         """Let the BeautifulSoup object know about namespaces encountered
         while parsing the document.
 
@@ -87,6 +125,7 @@ class LXMLTreeBuilderForXML(TreeBuilder):
 
         :param mapping: A dictionary mapping namespace prefixes to URIs.
         """
+        assert self.soup is not None
         for key, value in list(mapping.items()):
             # This is 'if key' and not 'if key is not None' because we
             # don't track un-prefixed namespaces. Soupselect will
@@ -98,19 +137,18 @@ class LXMLTreeBuilderForXML(TreeBuilder):
                 # prefix, the first one in the document takes precedence.
                 self.soup._namespaces[key] = value
                 
-    def default_parser(self, encoding):
+    def default_parser(self, encoding:Optional[_Encoding]) -> Type:
         """Find the default parser for the given encoding.
 
-        :param encoding: A string.
         :return: Either a parser object or a class, which
           will be instantiated with default arguments.
         """
         if self._default_parser is not None:
             return self._default_parser
-        return etree.XMLParser(
+        return self.DEFAULT_PARSER_CLASS(
             target=self, strip_cdata=False, recover=True, encoding=encoding)
 
-    def parser_for(self, encoding):
+    def parser_for(self, encoding: Optional[_Encoding]) -> Any:
         """Instantiate an appropriate parser for the given encoding.
 
         :param encoding: A string.
@@ -119,36 +157,39 @@ class LXMLTreeBuilderForXML(TreeBuilder):
         # Use the default parser.
         parser = self.default_parser(encoding)
 
-        if isinstance(parser, Callable):
+        if callable(parser):
             # Instantiate the parser with default arguments
             parser = parser(
                 target=self, strip_cdata=False, recover=True, encoding=encoding
             )
         return parser
 
-    def __init__(self, parser=None, empty_element_tags=None, **kwargs):
+    def __init__(self, parser:Optional[Any]=None,
+                 empty_element_tags:Optional[Set[str]]=None, **kwargs):
         # TODO: Issue a warning if parser is present but not a
         # callable, since that means there's no way to create new
         # parsers for different encodings.
         self._default_parser = parser
-        if empty_element_tags is not None:
-            self.empty_element_tags = set(empty_element_tags)
         self.soup = None
         self.nsmaps = [self.DEFAULT_NSMAPS_INVERTED]
         self.active_namespace_prefixes = [dict(self.DEFAULT_NSMAPS)]
         super(LXMLTreeBuilderForXML, self).__init__(**kwargs)
         
-    def _getNsTag(self, tag):
+    def _getNsTag(self, tag:str) -> Tuple[Optional[str], str]:
         # Split the namespace URL out of a fully-qualified lxml tag
         # name. Copied from lxml's src/lxml/sax.py.
         if tag[0] == '{':
-            return tuple(tag[1:].split('}', 1))
+            namespace, name = tag[1:].split('}', 1)
+            return (namespace, name)
         else:
             return (None, tag)
 
-    def prepare_markup(self, markup, user_specified_encoding=None,
-                       exclude_encodings=None,
-                       document_declared_encoding=None):
+    def prepare_markup(
+            self, markup:_RawMarkup,
+            user_specified_encoding:Optional[_Encoding]=None,
+            document_declared_encoding:Optional[_Encoding]=None,
+            exclude_encodings:Optional[_Encodings]=None,
+    ) -> Iterable[Tuple[Union[str,bytes], Optional[_Encoding], Optional[_Encoding], bool]]:
         """Run any preliminary steps necessary to make incoming markup
         acceptable to the parser.
 
@@ -166,13 +207,12 @@ class LXMLTreeBuilderForXML(TreeBuilder):
         :param exclude_encodings: The user asked _not_ to try any of
             these encodings.
 
-        :yield: A series of 4-tuples:
-         (markup, encoding, declared encoding,
-          has undergone character replacement)
+        :yield: A series of 4-tuples: (markup, encoding, declared encoding,
+            has undergone character replacement)
 
-         Each 4-tuple represents a strategy for converting the
-         document to Unicode and parsing it. Each strategy will be tried 
-         in turn.
+            Each 4-tuple represents a strategy for converting the
+            document to Unicode and parsing it. Each strategy will be tried 
+            in turn.
         """
         is_html = not self.is_xml
         if is_html:
@@ -200,14 +240,25 @@ class LXMLTreeBuilderForXML(TreeBuilder):
             yield (markup.encode("utf8"), "utf8",
                    document_declared_encoding, False)
 
-        # This was provided by the end-user; treat it as a known
-        # definite encoding per the algorithm laid out in the HTML5
-        # spec.  (See the EncodingDetector class for details.)
-        known_definite_encodings = [user_specified_encoding]
+            # Since the document was Unicode in the first place, there
+            # is no need to try any more strategies; we know this will
+            # work.
+            return
+
+        known_definite_encodings: List[_Encoding] = []
+        if user_specified_encoding:
+            # This was provided by the end-user; treat it as a known
+            # definite encoding per the algorithm laid out in the
+            # HTML5 spec. (See the EncodingDetector class for
+            # details.)
+            known_definite_encodings.append(user_specified_encoding)
+
+        user_encodings: List[_Encoding] = []
+        if document_declared_encoding:
+            # This was found in the document; treat it as a slightly
+            # lower-priority user encoding.
+            user_encodings.append(document_declared_encoding)
 
-        # This was found in the document; treat it as a slightly lower-priority
-        # user encoding.
-        user_encodings = [document_declared_encoding]
         detector = EncodingDetector(
             markup, known_definite_encodings=known_definite_encodings,
             user_encodings=user_encodings, is_html=is_html,
@@ -216,34 +267,45 @@ class LXMLTreeBuilderForXML(TreeBuilder):
         for encoding in detector.encodings:
             yield (detector.markup, encoding, document_declared_encoding, False)
 
-    def feed(self, markup):
+    def feed(self, markup:Union[bytes,str]) -> None:
+        io: IO
         if isinstance(markup, bytes):
-            markup = BytesIO(markup)
+            io = BytesIO(markup)
         elif isinstance(markup, str):
-            markup = StringIO(markup)
+            io = StringIO(markup)
 
+        # initialize_soup is called before feed, so we know this
+        # is not None.
+        assert self.soup is not None
+            
         # Call feed() at least once, even if the markup is empty,
         # or the parser won't be initialized.
-        data = markup.read(self.CHUNK_SIZE)
+        data = io.read(self.CHUNK_SIZE)
         try:
             self.parser = self.parser_for(self.soup.original_encoding)
             self.parser.feed(data)
             while len(data) != 0:
                 # Now call feed() on the rest of the data, chunk by chunk.
-                data = markup.read(self.CHUNK_SIZE)
+                data = io.read(self.CHUNK_SIZE)
                 if len(data) != 0:
                     self.parser.feed(data)
             self.parser.close()
         except (UnicodeDecodeError, LookupError, etree.ParserError) as e:
             raise ParserRejectedMarkup(e)
 
-    def close(self):
+    def close(self) -> None:
         self.nsmaps = [self.DEFAULT_NSMAPS_INVERTED]
 
-    def start(self, name, attrs, nsmap={}):
+    def start(self, name:str, attrs:Dict[str, str], nsmap:_NamespaceMapping={}):
+        # This is called by lxml code as a result of calling
+        # BeautifulSoup.feed(), and we know self.soup is set by the time feed()
+        # is called.
+        assert self.soup is not None
+        
         # Make sure attrs is a mutable dict--lxml may send an immutable dictproxy.
         attrs = dict(attrs)
-        nsprefix = None
+        nsprefix: Optional[_NamespacePrefix] = None
+        namespace: Optional[_NamespaceURL] = None
         # Invert each namespace map as it comes in.
         if len(nsmap) == 0 and len(self.nsmaps) > 1:
                 # There are no new namespaces for this tag, but
@@ -285,7 +347,7 @@ class LXMLTreeBuilderForXML(TreeBuilder):
         # Namespaces are in play. Find any attributes that came in
         # from lxml with namespaces attached to their names, and
         # turn then into NamespacedAttribute objects.
-        new_attrs = {}
+        new_attrs:Dict[Union[str,NamespacedAttribute], str] = {}
         for attr, value in list(attrs.items()):
             namespace, attr = self._getNsTag(attr)
             if namespace is None:
@@ -303,7 +365,7 @@ class LXMLTreeBuilderForXML(TreeBuilder):
             namespaces=self.active_namespace_prefixes[-1]
         )
         
-    def _prefix_for_namespace(self, namespace):
+    def _prefix_for_namespace(self, namespace:Optional[_NamespaceURL]) -> Optional[_NamespacePrefix]:
         """Find the currently active prefix for the given namespace."""
         if namespace is None:
             return None
@@ -312,7 +374,8 @@ class LXMLTreeBuilderForXML(TreeBuilder):
                 return inverted_nsmap[namespace]
         return None
 
-    def end(self, name):
+    def end(self, name:str) -> None:
+        assert self.soup is not None
         self.soup.endData()
         completed_tag = self.soup.tagStack[-1]
         namespace, name = self._getNsTag(name)
@@ -334,44 +397,49 @@ class LXMLTreeBuilderForXML(TreeBuilder):
                 # namespace prefixes.
                 self.active_namespace_prefixes.pop()
             
-    def pi(self, target, data):
+    def pi(self, target:str, data:str) -> None:
+        assert self.soup is not None
         self.soup.endData()
         data = target + ' ' + data
         self.soup.handle_data(data)
         self.soup.endData(self.processing_instruction_class)
         
-    def data(self, content):
+    def data(self, content:str) -> None:
+        assert self.soup is not None
         self.soup.handle_data(content)
 
-    def doctype(self, name, pubid, system):
+    def doctype(self, name:str, pubid:str, system:str) -> None:
+        assert self.soup is not None
         self.soup.endData()
         doctype = Doctype.for_name_and_ids(name, pubid, system)
         self.soup.object_was_parsed(doctype)
 
-    def comment(self, content):
+    def comment(self, content:str) -> None:
         "Handle comments as Comment objects."
+        assert self.soup is not None
         self.soup.endData()
         self.soup.handle_data(content)
         self.soup.endData(Comment)
 
-    def test_fragment_to_document(self, fragment):
+    def test_fragment_to_document(self, fragment:str) -> str:
         """See `TreeBuilder`."""
         return '<?xml version="1.0" encoding="utf-8"?>\n%s' % fragment
 
 
 class LXMLTreeBuilder(HTMLTreeBuilder, LXMLTreeBuilderForXML):
 
-    NAME = LXML
-    ALTERNATE_NAMES = ["lxml-html"]
+    NAME:str = LXML
+    ALTERNATE_NAMES: Iterable[str] = ["lxml-html"]
 
-    features = ALTERNATE_NAMES + [NAME, HTML, FAST, PERMISSIVE]
-    is_xml = False
-    processing_instruction_class = ProcessingInstruction
+    features: Iterable[str] = list(ALTERNATE_NAMES) + [NAME, HTML, FAST, PERMISSIVE]
+    is_xml: bool = False
 
-    def default_parser(self, encoding):
+    def default_parser(self, encoding:Optional[_Encoding]) -> Type[Any]:
         return etree.HTMLParser
 
-    def feed(self, markup):
+    def feed(self, markup:_RawMarkup) -> None: 
+        # We know self.soup is set by the time feed() is called.
+        assert self.soup is not None
         encoding = self.soup.original_encoding
         try:
             self.parser = self.parser_for(encoding)
@@ -381,6 +449,7 @@ class LXMLTreeBuilder(HTMLTreeBuilder, LXMLTreeBuilderForXML):
             raise ParserRejectedMarkup(e)
 
 
-    def test_fragment_to_document(self, fragment):
+    def test_fragment_to_document(self, fragment:str) -> str:
         """See `TreeBuilder`."""
         return '<html><body>%s</body></html>' % fragment
+
diff --git a/bs4/css.py b/bs4/css.py
index 245ac60..0477de8 100644
--- a/bs4/css.py
+++ b/bs4/css.py
@@ -1,6 +1,36 @@
-"""Integration code for CSS selectors using Soup Sieve (pypi: soupsieve)."""
-
+"""Integration code for CSS selectors using `Soup Sieve <https://facelessuser.github.io/soupsieve/>`_ (pypi: ``soupsieve``).
+
+Acquire a `CSS` object through the `bs4.element.Tag.css` attribute of
+the starting point of your CSS selector, or (if you want to run a
+selector against the entire document) of the `BeautifulSoup` object
+itself.
+
+The main advantage of doing this instead of using ``soupsieve``
+functions is that you don't need to keep passing the `bs4.element.Tag` to be
+selected against, since the `CSS` object is permanently scoped to that
+`bs4.element.Tag`.
+
+"""
+
+from __future__ import annotations
+
+from types import ModuleType
+from typing import (
+    Any,
+    cast,
+    Iterable,
+    Iterator,
+    Optional,
+    TYPE_CHECKING,
+)
 import warnings
+from bs4._typing import _NamespaceMapping
+if TYPE_CHECKING:
+    from soupsieve import SoupSieve
+    from bs4 import element
+    from bs4.element import ResultSet, Tag
+    
+soupsieve: Optional[ModuleType]
 try:
     import soupsieve
 except ImportError as e:
@@ -9,34 +39,22 @@ except ImportError as e:
         'The soupsieve package is not installed. CSS selectors cannot be used.'
     )
 
-
 class CSS(object):
-    """A proxy object against the soupsieve library, to simplify its
+    """A proxy object against the ``soupsieve`` library, to simplify its
     CSS selector API.
 
-    Acquire this object through the .css attribute on the
-    BeautifulSoup object, or on the Tag you want to use as the
-    starting point for a CSS selector.
-
-    The main advantage of doing this is that the tag to be selected
-    against doesn't need to be explicitly specified in the function
-    calls, since it's already scoped to a tag.
-    """
-
-    def __init__(self, tag, api=soupsieve):
-        """Constructor.
-
-        You don't need to instantiate this class yourself; instead,
-        access the .css attribute on the BeautifulSoup object, or on
-        the Tag you want to use as the starting point for your CSS
-        selector.
+    You don't need to instantiate this class yourself; instead, use
+    `element.Tag.css`.
 
-        :param tag: All CSS selectors will use this as their starting
-        point.
+    :param tag: All CSS selectors run by this object will use this as
+        their starting point.
 
-        :param api: A plug-in replacement for the soupsieve module,
-        designed mainly for use in tests.
-        """
+    :param api: An optional drop-in replacement for the ``soupsieve`` module,
+        intended for use in unit tests.
+    """
+    def __init__(self, tag: element.Tag, api:Optional[ModuleType]=None):
+        if api is None:
+            api = soupsieve
         if api is None:
             raise NotImplementedError(
                 "Cannot execute CSS selectors because the soupsieve package is not installed."
@@ -44,19 +62,19 @@ class CSS(object):
         self.api = api
         self.tag = tag
 
-    def escape(self, ident):
+    def escape(self, ident:str) -> str:
         """Escape a CSS identifier.
 
-        This is a simple wrapper around soupselect.escape(). See the
+        This is a simple wrapper around `soupsieve.escape() <https://facelessuser.github.io/soupsieve/api/#soupsieveescape>`_. See the
         documentation for that function for more information.
         """
         if soupsieve is None:
             raise NotImplementedError(
                 "Cannot escape CSS identifiers because the soupsieve package is not installed."
             )
-        return self.api.escape(ident)
+        return cast(str, self.api.escape(ident))
 
-    def _ns(self, ns, select):
+    def _ns(self, ns:Optional[_NamespaceMapping], select:str) -> Optional[_NamespaceMapping]:
         """Normalize a dictionary of namespaces."""
         if not isinstance(select, self.api.SoupSieve) and ns is None:
             # If the selector is a precompiled pattern, it already has
@@ -65,7 +83,7 @@ class CSS(object):
             ns = self.tag._namespaces
         return ns
 
-    def _rs(self, results):
+    def _rs(self, results:Iterable[Tag]) -> ResultSet[Tag]:
         """Normalize a list of results to a Resultset.
 
         A ResultSet is more consistent with the rest of Beautiful
@@ -77,7 +95,12 @@ class CSS(object):
         from bs4.element import ResultSet
         return ResultSet(None, results)
 
-    def compile(self, select, namespaces=None, flags=0, **kwargs):
+    def compile(self,
+                select:str,
+                namespaces:Optional[_NamespaceMapping]=None,
+                flags:int=0,
+                **kwargs:Any
+                ) -> SoupSieve:
         """Pre-compile a selector and return the compiled object.
 
         :param selector: A CSS selector.
@@ -88,10 +111,10 @@ class CSS(object):
            parsing the document.
 
         :param flags: Flags to be passed into Soup Sieve's
-            soupsieve.compile() method.
+            `soupsieve.compile() <https://facelessuser.github.io/soupsieve/api/#soupsievecompile>`_ method.
 
-        :param kwargs: Keyword arguments to be passed into SoupSieve's
-           soupsieve.compile() method.
+        :param kwargs: Keyword arguments to be passed into Soup Sieve's
+           `soupsieve.compile() <https://facelessuser.github.io/soupsieve/api/#soupsievecompile>`_ method.
 
         :return: A precompiled selector object.
         :rtype: soupsieve.SoupSieve
@@ -100,13 +123,16 @@ class CSS(object):
             select, self._ns(namespaces, select), flags, **kwargs
         )
 
-    def select_one(self, select, namespaces=None, flags=0, **kwargs):
+    def select_one(
+            self, select:str,
+            namespaces:Optional[_NamespaceMapping]=None,
+            flags:int=0, **kwargs:Any
+    )-> element.Tag | None:
         """Perform a CSS selection operation on the current Tag and return the
-        first result.
+        first result, if any.
 
         This uses the Soup Sieve library. For more information, see
-        that library's documentation for the soupsieve.select_one()
-        method.
+        that library's documentation for the `soupsieve.select_one() <https://facelessuser.github.io/soupsieve/api/#soupsieveselect_one>`_ method.
 
         :param selector: A CSS selector.
 
@@ -116,27 +142,24 @@ class CSS(object):
            parsing the document.
 
         :param flags: Flags to be passed into Soup Sieve's
-            soupsieve.select_one() method.
-
-        :param kwargs: Keyword arguments to be passed into SoupSieve's
-           soupsieve.select_one() method.
-
-        :return: A Tag, or None if the selector has no match.
-        :rtype: bs4.element.Tag
+            `soupsieve.select_one() <https://facelessuser.github.io/soupsieve/api/#soupsieveselect_one>`_ method.
 
+        :param kwargs: Keyword arguments to be passed into Soup Sieve's
+           `soupsieve.select_one() <https://facelessuser.github.io/soupsieve/api/#soupsieveselect_one>`_ method.
         """
         return self.api.select_one(
             select, self.tag, self._ns(namespaces, select), flags, **kwargs
         )
 
-    def select(self, select, namespaces=None, limit=0, flags=0, **kwargs):
-        """Perform a CSS selection operation on the current Tag.
+    def select(self, select:str,
+               namespaces:Optional[_NamespaceMapping]=None,
+               limit:int=0, flags:int=0, **kwargs:Any) -> ResultSet[Tag]:
+        """Perform a CSS selection operation on the current `element.Tag`.
 
         This uses the Soup Sieve library. For more information, see
-        that library's documentation for the soupsieve.select()
-        method.
+        that library's documentation for the `soupsieve.select() <https://facelessuser.github.io/soupsieve/api/#soupsieveselect>`_ method.
 
-        :param selector: A string containing a CSS selector.
+        :param selector: A CSS selector.
 
         :param namespaces: A dictionary mapping namespace prefixes
             used in the CSS selector to namespace URIs. By default,
@@ -146,14 +169,10 @@ class CSS(object):
         :param limit: After finding this number of results, stop looking.
 
         :param flags: Flags to be passed into Soup Sieve's
-            soupsieve.select() method.
-
-        :param kwargs: Keyword arguments to be passed into SoupSieve's
-            soupsieve.select() method.
-
-        :return: A ResultSet of Tag objects.
-        :rtype: bs4.element.ResultSet
+            `soupsieve.select() <https://facelessuser.github.io/soupsieve/api/#soupsieveselect>`_ method.
 
+        :param kwargs: Keyword arguments to be passed into Soup Sieve's
+           `soupsieve.select() <https://facelessuser.github.io/soupsieve/api/#soupsieveselect>`_ method.
         """
         if limit is None:
             limit = 0
@@ -165,11 +184,14 @@ class CSS(object):
             )
         )
 
-    def iselect(self, select, namespaces=None, limit=0, flags=0, **kwargs):
-        """Perform a CSS selection operation on the current Tag.
+    def iselect(self, select:str,
+                namespaces:Optional[_NamespaceMapping]=None,
+                limit:int=0, flags:int=0, **kwargs:Any) -> Iterator[element.Tag]:
+        """Perform a CSS selection operation on the current `element.Tag`.
 
         This uses the Soup Sieve library. For more information, see
-        that library's documentation for the soupsieve.iselect()
+        that library's documentation for the `soupsieve.iselect()
+        <https://facelessuser.github.io/soupsieve/api/#soupsieveiselect>`_
         method. It is the same as select(), but it returns a generator
         instead of a list.
 
@@ -183,23 +205,23 @@ class CSS(object):
         :param limit: After finding this number of results, stop looking.
 
         :param flags: Flags to be passed into Soup Sieve's
-            soupsieve.iselect() method.
-
-        :param kwargs: Keyword arguments to be passed into SoupSieve's
-            soupsieve.iselect() method.
+            `soupsieve.iselect() <https://facelessuser.github.io/soupsieve/api/#soupsieveiselect>`_ method.
 
-        :return: A generator
-        :rtype: types.GeneratorType
+        :param kwargs: Keyword arguments to be passed into Soup Sieve's
+           `soupsieve.iselect() <https://facelessuser.github.io/soupsieve/api/#soupsieveiselect>`_ method.
         """
         return self.api.iselect(
             select, self.tag, self._ns(namespaces, select), limit, flags, **kwargs
         )
 
-    def closest(self, select, namespaces=None, flags=0, **kwargs):
-        """Find the Tag closest to this one that matches the given selector.
+    def closest(self, select:str,
+               namespaces:Optional[_NamespaceMapping]=None,
+                flags:int=0, **kwargs:Any) -> Optional[element.Tag]:
+        """Find the `element.Tag` closest to this one that matches the given selector.
 
         This uses the Soup Sieve library. For more information, see
-        that library's documentation for the soupsieve.closest()
+        that library's documentation for the `soupsieve.closest()
+        <https://facelessuser.github.io/soupsieve/api/#soupsieveclosest>`_
         method.
 
         :param selector: A string containing a CSS selector.
@@ -210,24 +232,24 @@ class CSS(object):
             parsing the document.
 
         :param flags: Flags to be passed into Soup Sieve's
-            soupsieve.closest() method.
+            `soupsieve.closest() <https://facelessuser.github.io/soupsieve/api/#soupsieveclosest>`_ method.
 
-        :param kwargs: Keyword arguments to be passed into SoupSieve's
-            soupsieve.closest() method.
-
-        :return: A Tag, or None if there is no match.
-        :rtype: bs4.Tag
+        :param kwargs: Keyword arguments to be passed into Soup Sieve's
+           `soupsieve.closest() <https://facelessuser.github.io/soupsieve/api/#soupsieveclosest>`_ method.
 
         """
         return self.api.closest(
             select, self.tag, self._ns(namespaces, select), flags, **kwargs
         )
 
-    def match(self, select, namespaces=None, flags=0, **kwargs):
-        """Check whether this Tag matches the given CSS selector.
+    def match(self, select:str,
+              namespaces:Optional[_NamespaceMapping]=None,
+              flags:int=0, **kwargs:Any) -> bool:
+        """Check whether or not this `element.Tag` matches the given CSS selector.
 
         This uses the Soup Sieve library. For more information, see
-        that library's documentation for the soupsieve.match()
+        that library's documentation for the `soupsieve.match()
+        <https://facelessuser.github.io/soupsieve/api/#soupsievematch>`_
         method.
 
         :param: a CSS selector.
@@ -238,25 +260,30 @@ class CSS(object):
             parsing the document.
 
         :param flags: Flags to be passed into Soup Sieve's
-            soupsieve.match() method.
+            `soupsieve.match()
+            <https://facelessuser.github.io/soupsieve/api/#soupsievematch>`_
+            method.
 
         :param kwargs: Keyword arguments to be passed into SoupSieve's
-            soupsieve.match() method.
-
-        :return: True if this Tag matches the selector; False otherwise.
-        :rtype: bool
+            `soupsieve.match()
+            <https://facelessuser.github.io/soupsieve/api/#soupsievematch>`_
+            method.
         """
-        return self.api.match(
+        return cast(bool, self.api.match(
             select, self.tag, self._ns(namespaces, select), flags, **kwargs
-        )
+        ))
 
-    def filter(self, select, namespaces=None, flags=0, **kwargs):
-        """Filter this Tag's direct children based on the given CSS selector.
+    def filter(self, select:str,
+               namespaces:Optional[_NamespaceMapping]=None,
+               flags:int=0, **kwargs:Any) -> ResultSet[Tag]:
+        """Filter this `element.Tag`'s direct children based on the given CSS selector.
 
         This uses the Soup Sieve library. It works the same way as
-        passing this Tag into that library's soupsieve.filter()
-        method. More information, for more information see the
-        documentation for soupsieve.filter().
+        passing a `element.Tag` into that library's `soupsieve.filter()
+        <https://facelessuser.github.io/soupsieve/api/#soupsievefilter>`_
+        method. For more information, see the documentation for
+        `soupsieve.filter()
+        <https://facelessuser.github.io/soupsieve/api/#soupsievefilter>`_.
 
         :param namespaces: A dictionary mapping namespace prefixes
             used in the CSS selector to namespace URIs. By default,
@@ -264,17 +291,18 @@ class CSS(object):
             parsing the document.
 
         :param flags: Flags to be passed into Soup Sieve's
-            soupsieve.filter() method.
+            `soupsieve.filter()
+            <https://facelessuser.github.io/soupsieve/api/#soupsievefilter>`_
+            method.
 
         :param kwargs: Keyword arguments to be passed into SoupSieve's
-            soupsieve.filter() method.
-
-        :return: A ResultSet of Tag objects.
-        :rtype: bs4.element.ResultSet
-
+            `soupsieve.filter()
+            <https://facelessuser.github.io/soupsieve/api/#soupsievefilter>`_
+            method.
         """
         return self._rs(
             self.api.filter(
                 select, self.tag, self._ns(namespaces, select), flags, **kwargs
             )
         )
+
diff --git a/bs4/dammit.py b/bs4/dammit.py
index 692433c..8c1b631 100644
--- a/bs4/dammit.py
+++ b/bs4/dammit.py
@@ -2,9 +2,11 @@
 """Beautiful Soup bonus library: Unicode, Dammit
 
 This library converts a bytestream to Unicode through any means
-necessary. It is heavily based on code from Mark Pilgrim's Universal
-Feed Parser. It works best on XML and HTML, but it does not rewrite the
-XML or HTML to reflect a new encoding; that's the tree builder's job.
+necessary. It is heavily based on code from Mark Pilgrim's `Universal
+Feed Parser <https://pypi.org/project/feedparser/>`_. It works best on
+XML and HTML, but it does not rewrite the XML or HTML to reflect a new
+encoding; that's the job of `TreeBuilder`.
+
 """
 # Use of this source code is governed by the MIT license.
 __license__ = "MIT"
@@ -12,9 +14,31 @@ __license__ = "MIT"
 from html.entities import codepoint2name
 from collections import defaultdict
 import codecs
+from html.entities import html5
 import re
-import logging
+from logging import Logger, getLogger
 import string
+from types import ModuleType
+from typing import (
+    Dict,
+    Iterable,
+    Iterator,
+    List,
+    Optional,
+    Pattern,
+    Sequence,
+    Set,
+    Tuple,
+    Type,
+    Union,
+    cast,
+)
+from bs4._typing import (
+    _Encoding,
+    _Encodings,
+    _RawMarkup,
+)
+import warnings
 
 # Import a library to autodetect character encodings. We'll support
 # any of a number of libraries that all support the same API:
@@ -22,37 +46,41 @@ import string
 # * cchardet
 # * chardet
 # * charset-normalizer
-chardet_module = None
+chardet_module: Optional[ModuleType] = None
 try:
     #  PyPI package: cchardet
-    import cchardet as chardet_module
+    import cchardet
+    chardet_module = cchardet
 except ImportError:
     try:
         #  Debian package: python-chardet
         #  PyPI package: chardet
-        import chardet as chardet_module
+        import chardet
+        chardet_module = chardet
     except ImportError:
         try:
             # PyPI package: charset-normalizer
-            import charset_normalizer as chardet_module
+            import charset_normalizer
+            chardet_module = charset_normalizer
         except ImportError:
             # No chardet available.
-            chardet_module = None
+            pass
 
-if chardet_module:
-    def chardet_dammit(s):
-        if isinstance(s, str):
-            return None
-        return chardet_module.detect(s)['encoding']
-else:
-    def chardet_dammit(s):
+
+def _chardet_dammit(s:bytes) -> Optional[str]:
+    """Try as hard as possible to detect the encoding of a bytestring."""
+    if chardet_module is None or isinstance(s, str):
         return None
+    module = chardet_module
+    return module.detect(s)['encoding']
 
 # Build bytestring and Unicode versions of regular expressions for finding
 # a declared encoding inside an XML or HTML document.
-xml_encoding = '^\\s*<\\?.*encoding=[\'"](.*?)[\'"].*\\?>'
-html_meta = '<\\s*meta[^>]+charset\\s*=\\s*["\']?([^>]*?)[ /;\'">]'
-encoding_res = dict()
+xml_encoding:str = '^\\s*<\\?.*encoding=[\'"](.*?)[\'"].*\\?>' #: :meta private:
+html_meta:str = '<\\s*meta[^>]+charset\\s*=\\s*["\']?([^>]*?)[ /;\'">]' #: :meta private:
+
+# TODO: The Pattern type here could use more refinement, but it's tricky.
+encoding_res: Dict[Type, Dict[str, Pattern]] = dict()
 encoding_res[bytes] = {
     'html' : re.compile(html_meta.encode("ascii"), re.I),
     'xml' : re.compile(xml_encoding.encode("ascii"), re.I),
@@ -62,12 +90,29 @@ encoding_res[str] = {
     'xml' : re.compile(xml_encoding, re.I)
 }
 
-from html.entities import html5
-
 class EntitySubstitution(object):
     """The ability to substitute XML or HTML entities for certain characters."""
 
-    def _populate_class_variables():
+    #: A map of named HTML entities to the corresponding Unicode string.
+    #:
+    #: :meta hide-value:
+    HTML_ENTITY_TO_CHARACTER: Dict[str, str]
+    
+    #: A map of Unicode strings to the corresponding named HTML entities;
+    #: the inverse of HTML_ENTITY_TO_CHARACTER.
+    #:
+    #: :meta hide-value:
+    CHARACTER_TO_HTML_ENTITY: Dict[str, str]
+
+    #: A regular expression that matches any character (or, in rare
+    #: cases, pair of characters) that can be replaced with a named
+    #: HTML entity.
+    #:
+    #: :meta hide-value:
+    CHARACTER_TO_HTML_ENTITY_RE: Pattern[str]
+
+    @classmethod
+    def _populate_class_variables(cls) -> None:
         """Initialize variables used by this class to manage the plethora of
         HTML5 named entities.
 
@@ -184,11 +229,14 @@ class EntitySubstitution(object):
             character = chr(codepoint)
             unicode_to_name[character] = name
 
-        return unicode_to_name, name_to_unicode, re.compile(re_definition)
-    (CHARACTER_TO_HTML_ENTITY, HTML_ENTITY_TO_CHARACTER,
-     CHARACTER_TO_HTML_ENTITY_RE) = _populate_class_variables()
+        cls.CHARACTER_TO_HTML_ENTITY = unicode_to_name
+        cls.HTML_ENTITY_TO_CHARACTER = name_to_unicode
+        cls.CHARACTER_TO_HTML_ENTITY_RE = re.compile(re_definition)
 
-    CHARACTER_TO_XML_ENTITY = {
+    #: A map of Unicode strings to the corresponding named XML entities.
+    #:
+    #: :meta hide-value:
+    CHARACTER_TO_XML_ENTITY: Dict[str, str] = {
         "'": "apos",
         '"': "quot",
         "&": "amp",
@@ -196,28 +244,37 @@ class EntitySubstitution(object):
         ">": "gt",
         }
 
-    BARE_AMPERSAND_OR_BRACKET = re.compile("([<>]|"
-                                           "&(?!#\\d+;|#x[0-9a-fA-F]+;|\\w+;)"
-                                           ")")
-
-    AMPERSAND_OR_BRACKET = re.compile("([<>&])")
+    #: A regular expression matching an angle bracket or an ampersand that
+    #: is not part of an XML or HTML entity.
+    #:
+    #: :meta hide-value:
+    BARE_AMPERSAND_OR_BRACKET: Pattern[str] = re.compile(
+        "([<>]|"
+        "&(?!#\\d+;|#x[0-9a-fA-F]+;|\\w+;)"
+        ")"
+    )
+
+    #: A regular expression matching an angle bracket or an ampersand.
+    #:
+    #: :meta hide-value:    
+    AMPERSAND_OR_BRACKET: Pattern[str] = re.compile("([<>&])")
 
     @classmethod
-    def _substitute_html_entity(cls, matchobj):
+    def _substitute_html_entity(cls, matchobj:re.Match[str]) -> str:
         """Used with a regular expression to substitute the
         appropriate HTML entity for a special character string."""
         entity = cls.CHARACTER_TO_HTML_ENTITY.get(matchobj.group(0))
         return "&%s;" % entity
 
     @classmethod
-    def _substitute_xml_entity(cls, matchobj):
+    def _substitute_xml_entity(cls, matchobj:re.Match[str]) -> str:
         """Used with a regular expression to substitute the
         appropriate XML entity for a special character string."""
         entity = cls.CHARACTER_TO_XML_ENTITY[matchobj.group(0)]
         return "&%s;" % entity
 
     @classmethod
-    def quoted_attribute_value(self, value):
+    def quoted_attribute_value(cls, value: str) -> str:
         """Make a value into a quoted XML attribute, possibly escaping it.
 
          Most strings will be quoted using double quotes.
@@ -233,7 +290,10 @@ class EntitySubstitution(object):
          double quotes will be escaped, and the string will be quoted
          using double quotes.
 
-          Welcome to "Bob's Bar" -> "Welcome to &quot;Bob's bar&quot;
+          Welcome to "Bob's Bar" -> Welcome to &quot;Bob's bar&quot;
+
+        :param value: The XML attribute value to quote
+        :return: The quoted value
         """
         quote_with = '"'
         if '"' in value:
@@ -254,17 +314,22 @@ class EntitySubstitution(object):
         return quote_with + value + quote_with
 
     @classmethod
-    def substitute_xml(cls, value, make_quoted_attribute=False):
-        """Substitute XML entities for special XML characters.
+    def substitute_xml(cls, value:str, make_quoted_attribute:bool=False) -> str:
+        """Replace special XML characters with named XML entities.
+
+        The less-than sign will become &lt;, the greater-than sign
+        will become &gt;, and any ampersands will become &amp;. If you
+        want ampersands that seem to be part of an entity definition
+        to be left alone, use `substitute_xml_containing_entities`
+        instead.
 
-        :param value: A string to be substituted. The less-than sign
-          will become &lt;, the greater-than sign will become &gt;,
-          and any ampersands will become &amp;. If you want ampersands
-          that appear to be part of an entity definition to be left
-          alone, use substitute_xml_containing_entities() instead.
+        :param value: A string to be substituted.
 
         :param make_quoted_attribute: If True, then the string will be
          quoted, as befits an attribute value.
+
+        :return: A version of ``value`` with special characters replaced
+         with named entities.
         """
         # Escape angle brackets and ampersands.
         value = cls.AMPERSAND_OR_BRACKET.sub(
@@ -276,7 +341,7 @@ class EntitySubstitution(object):
 
     @classmethod
     def substitute_xml_containing_entities(
-        cls, value, make_quoted_attribute=False):
+        cls, value: str, make_quoted_attribute:bool=False) -> str:
         """Substitute XML entities for special XML characters.
 
         :param value: A string to be substituted. The less-than sign will
@@ -297,10 +362,10 @@ class EntitySubstitution(object):
         return value
 
     @classmethod
-    def substitute_html(cls, s):
+    def substitute_html(cls, s: str) -> str:
         """Replace certain Unicode characters with named HTML entities.
 
-        This differs from data.encode(encoding, 'xmlcharrefreplace')
+        This differs from ``data.encode(encoding, 'xmlcharrefreplace')``
         in that the goal is to make the result more readable (to those
         with ASCII displays) rather than to recover from
         errors. There's absolutely nothing wrong with a UTF-8 string
@@ -308,109 +373,126 @@ class EntitySubstitution(object):
         character with "&eacute;" will make it more readable to some
         people.
 
-        :param s: A Unicode string.
+        :param s: The string to be modified.
+        :return: The string with some Unicode characters replaced with
+           HTML entities.
         """
         return cls.CHARACTER_TO_HTML_ENTITY_RE.sub(
             cls._substitute_html_entity, s)
-
+EntitySubstitution._populate_class_variables()
 
 class EncodingDetector:
-    """Suggests a number of possible encodings for a bytestring.
+    """This class is capable of guessing a number of possible encodings
+    for a bytestring.
 
     Order of precedence:
 
     1. Encodings you specifically tell EncodingDetector to try first
-    (the known_definite_encodings argument to the constructor).
-
+       (the ``known_definite_encodings`` argument to the constructor).
+    
     2. An encoding determined by sniffing the document's byte-order mark.
-
+    
     3. Encodings you specifically tell EncodingDetector to try if
-    byte-order mark sniffing fails (the user_encodings argument to the
-    constructor).
+       byte-order mark sniffing fails (the ``user_encodings`` argument to the
+       constructor).
 
     4. An encoding declared within the bytestring itself, either in an
-    XML declaration (if the bytestring is to be interpreted as an XML
-    document), or in a <meta> tag (if the bytestring is to be
-    interpreted as an HTML document.)
+       XML declaration (if the bytestring is to be interpreted as an XML
+       document), or in a <meta> tag (if the bytestring is to be
+       interpreted as an HTML document.)
 
     5. An encoding detected through textual analysis by chardet,
-    cchardet, or a similar external library.
+       cchardet, or a similar external library.
 
-    4. UTF-8.
+    6. UTF-8.
 
-    5. Windows-1252.
+    7. Windows-1252.
 
-    """
-    def __init__(self, markup, known_definite_encodings=None,
-                 is_html=False, exclude_encodings=None,
-                 user_encodings=None, override_encodings=None):
-        """Constructor.
-
-        :param markup: Some markup in an unknown encoding.
-
-        :param known_definite_encodings: When determining the encoding
-            of `markup`, these encodings will be tried first, in
-            order. In HTML terms, this corresponds to the "known
-            definite encoding" step defined here:
-            https://html.spec.whatwg.org/multipage/parsing.html#parsing-with-a-known-character-encoding
-
-        :param user_encodings: These encodings will be tried after the
-            `known_definite_encodings` have been tried and failed, and
-            after an attempt to sniff the encoding by looking at a
-            byte order mark has failed. In HTML terms, this
-            corresponds to the step "user has explicitly instructed
-            the user agent to override the document's character
-            encoding", defined here:
-            https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding
-
-        :param override_encodings: A deprecated alias for
-            known_definite_encodings. Any encodings here will be tried
-            immediately after the encodings in
-            known_definite_encodings.
-
-        :param is_html: If True, this markup is considered to be
-            HTML. Otherwise it's assumed to be XML.
-
-        :param exclude_encodings: These encodings will not be tried,
-            even if they otherwise would be.
+    :param markup: Some markup in an unknown encoding.
 
-        """
+    :param known_definite_encodings: When determining the encoding
+        of ``markup``, these encodings will be tried first, in
+        order. In HTML terms, this corresponds to the "known
+        definite encoding" step defined in `section 13.2.3.1 of the HTML standard <https://html.spec.whatwg.org/multipage/parsing.html#parsing-with-a-known-character-encoding>`_.
+
+    :param user_encodings: These encodings will be tried after the
+        ``known_definite_encodings`` have been tried and failed, and
+        after an attempt to sniff the encoding by looking at a
+        byte order mark has failed. In HTML terms, this
+        corresponds to the step "user has explicitly instructed
+        the user agent to override the document's character
+        encoding", defined in `section 13.2.3.2 of the HTML standard <https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding>`_.
+
+    :param override_encodings: A **deprecated** alias for
+        ``known_definite_encodings``. Any encodings here will be tried
+        immediately after the encodings in
+        ``known_definite_encodings``.
+
+    :param is_html: If True, this markup is considered to be 
+        HTML. Otherwise it's assumed to be XML.
+
+    :param exclude_encodings: These encodings will not be tried,
+        even if they otherwise would be.
+
+    """   
+    def __init__(self, markup:bytes,
+                 known_definite_encodings:Optional[_Encodings]=None,
+                 is_html:Optional[bool]=False,
+                 exclude_encodings:Optional[_Encodings]=None,
+                 user_encodings:Optional[_Encodings]=None,
+                 override_encodings:Optional[_Encodings]=None):
         self.known_definite_encodings = list(known_definite_encodings or [])
         if override_encodings:
+            warnings.warn(
+                "The 'override_encodings' argument was deprecated in 4.10.0. Use 'known_definite_encodings' instead.",
+                DeprecationWarning,
+                stacklevel=3
+            )
             self.known_definite_encodings += override_encodings
         self.user_encodings = user_encodings or []
         exclude_encodings = exclude_encodings or []
         self.exclude_encodings = set([x.lower() for x in exclude_encodings])
         self.chardet_encoding = None
-        self.is_html = is_html
-        self.declared_encoding = None
+        self.is_html = False if is_html is None else is_html
+        self.declared_encoding: Optional[str] = None
 
         # First order of business: strip a byte-order mark.
         self.markup, self.sniffed_encoding = self.strip_byte_order_mark(markup)
 
-    def _usable(self, encoding, tried):
+    known_definite_encodings:_Encodings
+    user_encodings:_Encodings
+    exclude_encodings:_Encodings
+    chardet_encoding:Optional[_Encoding]
+    is_html:bool
+    declared_encoding:Optional[_Encoding]
+    markup:bytes
+    sniffed_encoding:Optional[_Encoding]
+        
+    def _usable(self, encoding:Optional[_Encoding], tried:Set[_Encoding]) -> bool:
         """Should we even bother to try this encoding?
 
         :param encoding: Name of an encoding.
-        :param tried: Encodings that have already been tried. This will be modified
-            as a side effect.
+        :param tried: Encodings that have already been tried. This
+            will be modified as a side effect.
         """
-        if encoding is not None:
-            encoding = encoding.lower()
-            if encoding in self.exclude_encodings:
-                return False
-            if encoding not in tried:
-                tried.add(encoding)
-                return True
+        if encoding is None:
+            return False
+        encoding = encoding.lower()
+        if encoding in self.exclude_encodings:
+            return False
+        if encoding not in tried:
+            tried.add(encoding)
+            return True
         return False
 
     @property
-    def encodings(self):
+    def encodings(self) -> Iterator[_Encoding]:
         """Yield a number of encodings that might work for this markup.
 
-        :yield: A sequence of strings.
+        :yield: A sequence of strings. Each is the name of an encoding
+           that *might* work to convert a bytestring into Unicode.
         """
-        tried = set()
+        tried:Set[_Encoding] = set()
 
         # First, try the known definite encodings
         for e in self.known_definite_encodings:
@@ -419,7 +501,9 @@ class EncodingDetector:
 
         # Did the document originally start with a byte-order mark
         # that indicated its encoding?
-        if self._usable(self.sniffed_encoding, tried):
+        if self.sniffed_encoding is not None and self._usable(
+            self.sniffed_encoding, tried
+        ):
             yield self.sniffed_encoding
 
         # Sniffing the byte-order mark did nothing; try the user
@@ -433,14 +517,18 @@ class EncodingDetector:
         if self.declared_encoding is None:
             self.declared_encoding = self.find_declared_encoding(
                 self.markup, self.is_html)
-        if self._usable(self.declared_encoding, tried):
+        if self.declared_encoding is not None and self._usable(
+            self.declared_encoding, tried
+        ):
             yield self.declared_encoding
 
         # Use third-party character set detection to guess at the
         # encoding.
         if self.chardet_encoding is None:
-            self.chardet_encoding = chardet_dammit(self.markup)
-        if self._usable(self.chardet_encoding, tried):
+            self.chardet_encoding = _chardet_dammit(self.markup)
+        if self.chardet_encoding is not None and self._usable(
+            self.chardet_encoding, tried
+        ):
             yield self.chardet_encoding
 
         # As a last-ditch effort, try utf-8 and windows-1252.
@@ -449,22 +537,24 @@ class EncodingDetector:
                 yield e
 
     @classmethod
-    def strip_byte_order_mark(cls, data):
+    def strip_byte_order_mark(cls, data:bytes) -> Tuple[bytes, Optional[_Encoding]]:
         """If a byte-order mark is present, strip it and return the encoding it implies.
 
-        :param data: Some markup.
-        :return: A 2-tuple (modified data, implied encoding)
+        :param data: A bytestring that may or may not begin with a
+           byte-order mark.
+
+        :return: A 2-tuple (data stripped of byte-order mark, encoding implied by byte-order mark)
         """
         encoding = None
         if isinstance(data, str):
             # Unicode data cannot have a byte-order mark.
             return data, encoding
         if (len(data) >= 4) and (data[:2] == b'\xfe\xff') \
-               and (data[2:4] != '\x00\x00'):
+               and (data[2:4] != b'\x00\x00'):
             encoding = 'utf-16be'
             data = data[2:]
         elif (len(data) >= 4) and (data[:2] == b'\xff\xfe') \
-                 and (data[2:4] != '\x00\x00'):
+                 and (data[2:4] != b'\x00\x00'):
             encoding = 'utf-16le'
             data = data[2:]
         elif data[:3] == b'\xef\xbb\xbf':
@@ -479,8 +569,9 @@ class EncodingDetector:
         return data, encoding
 
     @classmethod
-    def find_declared_encoding(cls, markup, is_html=False, search_entire_document=False):
-        """Given a document, tries to find its declared encoding.
+    def find_declared_encoding(cls, markup:Union[bytes,str], is_html:bool=False, search_entire_document:bool=False) -> Optional[_Encoding]:
+        """Given a document, tries to find an encoding declared within the
+        text of the document itself.
 
         An XML encoding is declared at the beginning of the document.
 
@@ -490,9 +581,12 @@ class EncodingDetector:
         :param markup: Some markup.
         :param is_html: If True, this markup is considered to be HTML. Otherwise
             it's assumed to be XML.
-        :param search_entire_document: Since an encoding is supposed to declared near the beginning
-            of the document, most of the time it's only necessary to search a few kilobytes of data.
-            Set this to True to force this method to search the entire document.
+        :param search_entire_document: Since an encoding is supposed
+            to declared near the beginning of the document, most of
+            the time it's only necessary to search a few kilobytes of
+            data.  Set this to True to force this method to search the
+            entire document.
+        :return: The declared encoding, if one is found.
         """
         if search_entire_document:
             xml_endpos = html_endpos = len(markup)
@@ -520,74 +614,69 @@ class EncodingDetector:
         return None
 
 class UnicodeDammit:
-    """A class for detecting the encoding of a *ML document and
-    converting it to a Unicode string. If the source encoding is
-    windows-1252, can replace MS smart quotes with their HTML or XML
-    equivalents."""
-
-    # This dictionary maps commonly seen values for "charset" in HTML
-    # meta tags to the corresponding Python codec names. It only covers
-    # values that aren't in Python's aliases and can't be determined
-    # by the heuristics in find_codec.
-    CHARSET_ALIASES = {"macintosh": "mac-roman",
-                       "x-sjis": "shift-jis"}
-
-    ENCODINGS_WITH_SMART_QUOTES = [
-        "windows-1252",
-        "iso-8859-1",
-        "iso-8859-2",
-        ]
+    """A class for detecting the encoding of a bytestring containing an
+    HTML or XML document, and decoding it to Unicode. If the source
+    encoding is windows-1252, `UnicodeDammit` can also replace
+    Microsoft smart quotes with their HTML or XML equivalents.
+
+    :param markup: HTML or XML markup in an unknown encoding.
+
+    :param known_definite_encodings: When determining the encoding
+        of ``markup``, these encodings will be tried first, in
+        order. In HTML terms, this corresponds to the "known
+        definite encoding" step defined in `section 13.2.3.1 of the HTML standard <https://html.spec.whatwg.org/multipage/parsing.html#parsing-with-a-known-character-encoding>`_.
+
+    :param user_encodings: These encodings will be tried after the
+        ``known_definite_encodings`` have been tried and failed, and
+        after an attempt to sniff the encoding by looking at a
+        byte order mark has failed. In HTML terms, this
+        corresponds to the step "user has explicitly instructed
+        the user agent to override the document's character
+        encoding", defined in `section 13.2.3.2 of the HTML standard <https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding>`_.
+
+    :param override_encodings: A **deprecated** alias for
+        ``known_definite_encodings``. Any encodings here will be tried
+        immediately after the encodings in
+        ``known_definite_encodings``.
+
+    :param smart_quotes_to: By default, Microsoft smart quotes will,
+       like all other characters, be converted to Unicode
+       characters. Setting this to ``ascii`` will convert them to ASCII
+       quotes instead.  Setting it to ``xml`` will convert them to XML
+       entity references, and setting it to ``html`` will convert them
+       to HTML entity references.
+
+    :param is_html: If True, ``markup`` is treated as an HTML
+       document. Otherwise it's treated as an XML document.
+
+    :param exclude_encodings: These encodings will not be considered,
+       even if the sniffing code thinks they might make sense.
 
-    def __init__(self, markup, known_definite_encodings=[],
-                 smart_quotes_to=None, is_html=False, exclude_encodings=[],
-                 user_encodings=None, override_encodings=None
+    """
+    def __init__(
+            self, markup:bytes,
+            known_definite_encodings:Optional[_Encodings]=[],
+            # TODO PYTHON 3.8 Literal is added to the typing module
+            #
+            # smart_quotes_to: Literal["ascii", "xml", "html"] | None = None,
+            smart_quotes_to: Optional[str] = None,
+            is_html: bool = False,
+            exclude_encodings:Optional[_Encodings] = [],
+            user_encodings:Optional[_Encodings] = None,
+            override_encodings:Optional[_Encodings] = None
     ):
-        """Constructor.
-
-        :param markup: A bytestring representing markup in an unknown encoding.
-
-        :param known_definite_encodings: When determining the encoding
-            of `markup`, these encodings will be tried first, in
-            order. In HTML terms, this corresponds to the "known
-            definite encoding" step defined here:
-            https://html.spec.whatwg.org/multipage/parsing.html#parsing-with-a-known-character-encoding
-
-        :param user_encodings: These encodings will be tried after the
-            `known_definite_encodings` have been tried and failed, and
-            after an attempt to sniff the encoding by looking at a
-            byte order mark has failed. In HTML terms, this
-            corresponds to the step "user has explicitly instructed
-            the user agent to override the document's character
-            encoding", defined here:
-            https://html.spec.whatwg.org/multipage/parsing.html#determining-the-character-encoding
-
-        :param override_encodings: A deprecated alias for
-            known_definite_encodings. Any encodings here will be tried
-            immediately after the encodings in
-            known_definite_encodings.
-
-        :param smart_quotes_to: By default, Microsoft smart quotes will, like all other characters, be converted
-           to Unicode characters. Setting this to 'ascii' will convert them to ASCII quotes instead.
-           Setting it to 'xml' will convert them to XML entity references, and setting it to 'html'
-           will convert them to HTML entity references.
-        :param is_html: If True, this markup is considered to be HTML. Otherwise
-            it's assumed to be XML.
-        :param exclude_encodings: These encodings will not be considered, even
-            if the sniffing code thinks they might make sense.
-
-        """
         self.smart_quotes_to = smart_quotes_to
         self.tried_encodings = []
         self.contains_replacement_characters = False
         self.is_html = is_html
-        self.log = logging.getLogger(__name__)
+        self.log = getLogger(__name__)
         self.detector = EncodingDetector(
             markup, known_definite_encodings, is_html, exclude_encodings,
             user_encodings, override_encodings
         )
 
         # Short-circuit if the data is in Unicode to begin with.
-        if isinstance(markup, str) or markup == '':
+        if isinstance(markup, str) or markup == b'':
             self.markup = markup
             self.unicode_markup = str(markup)
             self.original_encoding = None
@@ -616,41 +705,117 @@ class UnicodeDammit:
                             "Some characters could not be decoded, and were "
                             "replaced with REPLACEMENT CHARACTER."
                     )
+
                     self.contains_replacement_characters = True
                     break
 
         # If none of that worked, we could at this point force it to
         # ASCII, but that would destroy so much data that I think
         # giving up is better.
-        self.unicode_markup = u
-        if not u:
+        #
+        # Note that this is extremely unlikely, probably impossible,
+        # because the "replace" strategy is so powerful. Even running
+        # the Python binary through Unicode, Dammit gives you Unicode,
+        # albeit Unicode riddled with REPLACEMENT CHARACTER.
+        if u is None:
             self.original_encoding = None
+            self.unicode_markup = None
+        else:
+            self.unicode_markup = u
+
+    #: The original markup, before it was converted to Unicode.
+    #: This is not necessarily the same as what was passed in to the
+    #: constructor, since any byte-order mark will be stripped.
+    markup:bytes
 
-    def _sub_ms_char(self, match):
+    #: The Unicode version of the markup, following conversion. This
+    #: is set to `None` if there was simply no way to convert the
+    #: bytestring to Unicode (as with binary data).
+    unicode_markup:Optional[str]
+
+    #: This is True if `UnicodeDammit.unicode_markup` contains
+    #: U+FFFD REPLACEMENT_CHARACTER characters which were not present
+    #: in `UnicodeDammit.markup`. These mark character sequences that
+    #: could not be represented in Unicode.
+    contains_replacement_characters: bool
+
+    #: Unicode, Dammit's best guess as to the original character
+    #: encoding of `UnicodeDammit.markup`.
+    original_encoding:Optional[_Encoding]
+
+    #: The strategy used to handle Microsoft smart quotes.
+    smart_quotes_to: Optional[str]
+
+    #: The (encoding, error handling strategy) 2-tuples that were used to
+    #: try and convert the markup to Unicode.
+    tried_encodings: List[Tuple[_Encoding, str]]
+
+    log: Logger #: :meta private:
+            
+    def _sub_ms_char(self, match:re.Match[bytes]) -> bytes:
         """Changes a MS smart quote character to an XML or HTML
-        entity, or an ASCII character."""
-        orig = match.group(1)
+        entity, or an ASCII character.
+
+        TODO: Since this is only used to convert smart quotes, it
+        could be simplified, and MS_CHARS_TO_ASCII made much less
+        parochial.
+        """
+        orig: bytes = match.group(1)
+        sub: bytes
         if self.smart_quotes_to == 'ascii':
-            sub = self.MS_CHARS_TO_ASCII.get(orig).encode()
+            if orig in self.MS_CHARS_TO_ASCII:
+                sub = self.MS_CHARS_TO_ASCII[orig].encode()
+            else:
+                # Shouldn't happen; substitute the character
+                # with itself.
+                sub = orig
         else:
-            sub = self.MS_CHARS.get(orig)
-            if type(sub) == tuple:
-                if self.smart_quotes_to == 'xml':
-                    sub = '&#x'.encode() + sub[1].encode() + ';'.encode()
+            if orig in self.MS_CHARS:
+                substitutions = self.MS_CHARS[orig]
+                if type(substitutions) == tuple:
+                    if self.smart_quotes_to == 'xml':
+                        sub = b'&#x' + substitutions[1].encode() + b';'
+                    else:
+                        sub = b'&' + substitutions[0].encode() + b';'
                 else:
-                    sub = '&'.encode() + sub[0].encode() + ';'.encode()
+                    substitutions = cast(str, substitutions)
+                    sub = substitutions.encode()
             else:
-                sub = sub.encode()
+                # Shouldn't happen; substitute the character
+                # for itself.
+                sub = orig
         return sub
+    
+    #: This dictionary maps commonly seen values for "charset" in HTML
+    #: meta tags to the corresponding Python codec names. It only covers
+    #: values that aren't in Python's aliases and can't be determined
+    #: by the heuristics in `find_codec`.
+    #:
+    #: :meta hide-value:
+    CHARSET_ALIASES: Dict[str, _Encoding] = {"macintosh": "mac-roman",
+                                             "x-sjis": "shift-jis"}
+
+    #: A list of encodings that tend to contain Microsoft smart quotes.
+    #:
+    #: :meta hide-value:
+    ENCODINGS_WITH_SMART_QUOTES: _Encodings = [
+        "windows-1252",
+        "iso-8859-1",
+        "iso-8859-2",
+        ]
 
-    def _convert_from(self, proposed, errors="strict"):
+    def _convert_from(self, proposed:_Encoding, errors:str="strict") -> Optional[str]:
         """Attempt to convert the markup to the proposed encoding.
 
         :param proposed: The name of a character encoding.
+        :param errors: An error handling strategy, used when calling `str`.
+        :return: The converted markup, or `None` if the proposed
+           encoding/error handling strategy didn't work.
         """
-        proposed = self.find_codec(proposed)
-        if not proposed or (proposed, errors) in self.tried_encodings:
+        lookup_result = self.find_codec(proposed)
+        if lookup_result is None or (lookup_result, errors) in self.tried_encodings:
             return None
+        proposed = lookup_result
         self.tried_encodings.append((proposed, errors))
         markup = self.markup
         # Convert smart quotes to HTML if coming from an encoding
@@ -665,36 +830,37 @@ class UnicodeDammit:
             #print("Trying to convert document to %s (errors=%s)" % (
             #    proposed, errors))
             u = self._to_unicode(markup, proposed, errors)
-            self.markup = u
+            self.unicode_markup = u
             self.original_encoding = proposed
         except Exception as e:
             #print("That didn't work!")
             #print(e)
             return None
         #print("Correct encoding: %s" % proposed)
-        return self.markup
+        return self.unicode_markup
 
-    def _to_unicode(self, data, encoding, errors="strict"):
-        """Given a string and its encoding, decodes the string into Unicode.
+    def _to_unicode(self, data:bytes, encoding:_Encoding, errors:str="strict") -> str:
+        """Given a bytestring and its encoding, decodes the string into Unicode.
 
         :param encoding: The name of an encoding.
+        :param errors: An error handling strategy, used when calling `str`.
         """
         return str(data, encoding, errors)
 
     @property
-    def declared_html_encoding(self):
-        """If the markup is an HTML document, returns the encoding declared _within_
-        the document.
+    def declared_html_encoding(self) -> Optional[str]:
+        """If the markup is an HTML document, returns the encoding, if any,
+        declared *inside* the document.
         """
         if not self.is_html:
             return None
         return self.detector.declared_encoding
 
-    def find_codec(self, charset):
-        """Convert the name of a character set to a codec name.
+    def find_codec(self, charset:_Encoding) -> Optional[str]:
+        """Look up the Python codec corresponding to a given character set.
 
         :param charset: The name of a character set.
-        :return: The name of a codec.
+        :return: The name of a Python codec.
         """
         value = (self._codec(self.CHARSET_ALIASES.get(charset, charset))
                or (charset and self._codec(charset.replace("-", "")))
@@ -706,7 +872,7 @@ class UnicodeDammit:
             return value.lower()
         return None
 
-    def _codec(self, charset):
+    def _codec(self, charset:_Encoding) -> Optional[str]:
         if not charset:
             return charset
         codec = None
@@ -718,8 +884,11 @@ class UnicodeDammit:
         return codec
 
 
-    # A partial mapping of ISO-Latin-1 to HTML entities/XML numeric entities.
-    MS_CHARS = {b'\x80': ('euro', '20AC'),
+    #: A partial mapping of ISO-Latin-1 to HTML entities/XML numeric entities.
+    #:
+    #: :meta hide-value:
+    MS_CHARS: Dict[bytes, Union[str, Tuple[str, str]]] = {
+        b'\x80': ('euro', '20AC'),
                 b'\x81': ' ',
                 b'\x82': ('sbquo', '201A'),
                 b'\x83': ('fnof', '192'),
@@ -752,10 +921,15 @@ class UnicodeDammit:
                 b'\x9e': ('#x17E', '17E'),
                 b'\x9f': ('Yuml', ''),}
 
-    # A parochial partial mapping of ISO-Latin-1 to ASCII. Contains
-    # horrors like stripping diacritical marks to turn á into a, but also
-    # contains non-horrors like turning “ into ".
-    MS_CHARS_TO_ASCII = {
+    #: A parochial partial mapping of ISO-Latin-1 to ASCII. Contains
+    #: horrors like stripping diacritical marks to turn á into a, but also
+    #: contains non-horrors like turning “ into ".
+    #:
+    #: Seriously, don't use this for anything other than removing smart
+    #: quotes.
+    #:
+    #: :meta private:
+    MS_CHARS_TO_ASCII: Dict[bytes, str] = {
         b'\x80' : 'EUR',
         b'\x81' : ' ',
         b'\x82' : ',',
@@ -809,7 +983,7 @@ class UnicodeDammit:
         b'\xb1' : '+-',
         b'\xb2' : '2',
         b'\xb3' : '3',
-        b'\xb4' : ("'", 'acute'),
+        b'\xb4' : "'",
         b'\xb5' : 'u',
         b'\xb6' : 'P',
         b'\xb7' : '*',
@@ -887,12 +1061,14 @@ class UnicodeDammit:
         b'\xff' : 'y',
         }
 
-    # A map used when removing rogue Windows-1252/ISO-8859-1
-    # characters in otherwise UTF-8 documents.
-    #
-    # Note that \x81, \x8d, \x8f, \x90, and \x9d are undefined in
-    # Windows-1252.
-    WINDOWS_1252_TO_UTF8 = {
+    #: A map used when removing rogue Windows-1252/ISO-8859-1
+    #: characters in otherwise UTF-8 documents.
+    #:
+    #: Note that \\x81, \\x8d, \\x8f, \\x90, and \\x9d are undefined in
+    #: Windows-1252.
+    #:
+    #: :meta hide-value:
+    WINDOWS_1252_TO_UTF8: Dict[int, bytes] = {
         0x80 : b'\xe2\x82\xac', # €
         0x82 : b'\xe2\x80\x9a', # ‚
         0x83 : b'\xc6\x92',     # ƒ
@@ -1017,33 +1193,37 @@ class UnicodeDammit:
         0xfe : b'\xc3\xbe',     # þ
         }
 
-    MULTIBYTE_MARKERS_AND_SIZES = [
+    #: :meta private:
+    MULTIBYTE_MARKERS_AND_SIZES:List[Tuple[int, int, int]] = [
         (0xc2, 0xdf, 2), # 2-byte characters start with a byte C2-DF
         (0xe0, 0xef, 3), # 3-byte characters start with E0-EF
         (0xf0, 0xf4, 4), # 4-byte characters start with F0-F4
         ]
 
-    FIRST_MULTIBYTE_MARKER = MULTIBYTE_MARKERS_AND_SIZES[0][0]
-    LAST_MULTIBYTE_MARKER = MULTIBYTE_MARKERS_AND_SIZES[-1][1]
+    #: :meta private:
+    FIRST_MULTIBYTE_MARKER:int = MULTIBYTE_MARKERS_AND_SIZES[0][0]
+
+    #: :meta private:
+    LAST_MULTIBYTE_MARKER:int = MULTIBYTE_MARKERS_AND_SIZES[-1][1]
 
     @classmethod
-    def detwingle(cls, in_bytes, main_encoding="utf8",
-                  embedded_encoding="windows-1252"):
+    def detwingle(cls, in_bytes:bytes, main_encoding:_Encoding="utf8",
+                  embedded_encoding:_Encoding="windows-1252") -> bytes:
         """Fix characters from one encoding embedded in some other encoding.
 
         Currently the only situation supported is Windows-1252 (or its
         subset ISO-8859-1), embedded in UTF-8.
 
         :param in_bytes: A bytestring that you suspect contains
-            characters from multiple encodings. Note that this _must_
+            characters from multiple encodings. Note that this *must*
             be a bytestring. If you've already converted the document
             to Unicode, you're too late.
-        :param main_encoding: The primary encoding of `in_bytes`.
+        :param main_encoding: The primary encoding of ``in_bytes``.
         :param embedded_encoding: The encoding that was used to embed characters
             in the main document.
-        :return: A bytestring in which `embedded_encoding`
-          characters have been converted to their `main_encoding`
-          equivalents.
+        :return: A bytestring similar to ``in_bytes``, in which
+          ``embedded_encoding`` characters have been converted to
+          their ``main_encoding`` equivalents.
         """
         if embedded_encoding.replace('_', '-').lower() not in (
             'windows-1252', 'windows_1252'):
@@ -1061,9 +1241,6 @@ class UnicodeDammit:
         pos = 0
         while pos < len(in_bytes):
             byte = in_bytes[pos]
-            if not isinstance(byte, int):
-                # Python 2.x
-                byte = ord(byte)
             if (byte >= cls.FIRST_MULTIBYTE_MARKER
                 and byte <= cls.LAST_MULTIBYTE_MARKER):
                 # This is the start of a UTF-8 multibyte character. Skip
diff --git a/bs4/diagnose.py b/bs4/diagnose.py
index e079772..201b879 100644
--- a/bs4/diagnose.py
+++ b/bs4/diagnose.py
@@ -7,8 +7,11 @@ import cProfile
 from io import BytesIO
 from html.parser import HTMLParser
 import bs4
-from bs4 import BeautifulSoup, __version__
+from bs4 import BeautifulSoup, __version__ 
 from bs4.builder import builder_registry
+from typing import TYPE_CHECKING
+if TYPE_CHECKING:
+    from bs4._typing import _IncomingMarkup
 
 import os
 import pstats
@@ -19,10 +22,10 @@ import traceback
 import sys
 import cProfile
 
-def diagnose(data):
+def diagnose(data:_IncomingMarkup) -> None:
     """Diagnostic suite for isolating common problems.
 
-    :param data: A string containing markup that needs to be explained.
+    :param data: Some markup that needs to be explained.
     :return: None; diagnostics are printed to standard output.
     """
     print(("Diagnostic running on Beautiful Soup %s" % __version__))
@@ -75,7 +78,7 @@ def diagnose(data):
 
         print(("-" * 80))
 
-def lxml_trace(data, html=True, **kwargs):
+def lxml_trace(data, html:bool=True, **kwargs) -> None:
     """Print out the lxml events that occur during parsing.
 
     This lets you see how lxml parses a document when no Beautiful
@@ -109,7 +112,7 @@ class AnnouncingParser(HTMLParser):
         print(s)
 
     def handle_starttag(self, name, attrs):
-        self._p("%s START" % name)
+        self._p(f"{name} {attrs} START")
 
     def handle_endtag(self, name):
         self._p("%s END" % name)
@@ -146,11 +149,14 @@ def htmlparser_trace(data):
     parser = AnnouncingParser()
     parser.feed(data)
 
-_vowels = "aeiou"
-_consonants = "bcdfghjklmnpqrstvwxyz"
+_vowels:str = "aeiou"
+_consonants:str = "bcdfghjklmnpqrstvwxyz"
 
-def rword(length=5):
-    "Generate a random word-like string."
+def rword(length:int=5) -> str:
+    """Generate a random word-like string.
+
+    :meta private:
+    """
     s = ''
     for i in range(length):
         if i % 2 == 0:
@@ -160,12 +166,18 @@ def rword(length=5):
         s += random.choice(t)
     return s
 
-def rsentence(length=4):
-    "Generate a random sentence-like string."
+def rsentence(length:int=4) -> str:
+    """Generate a random sentence-like string.
+
+    :meta private:
+    """
     return " ".join(rword(random.randint(4,9)) for i in range(length))
         
-def rdoc(num_elements=1000):
-    """Randomly generate an invalid HTML document."""
+def rdoc(num_elements:int=1000) -> str:
+    """Randomly generate an invalid HTML document.
+
+    :meta private:
+    """
     tag_names = ['p', 'div', 'span', 'i', 'b', 'script', 'table']
     elements = []
     for i in range(num_elements):
@@ -182,24 +194,24 @@ def rdoc(num_elements=1000):
             elements.append("</%s>" % tag_name)
     return "<html>" + "\n".join(elements) + "</html>"
 
-def benchmark_parsers(num_elements=100000):
+def benchmark_parsers(num_elements:int=100000) -> None:
     """Very basic head-to-head performance benchmark."""
     print(("Comparative parser benchmark on Beautiful Soup %s" % __version__))
     data = rdoc(num_elements)
     print(("Generated a large invalid HTML document (%d bytes)." % len(data)))
     
-    for parser in ["lxml", ["lxml", "html"], "html5lib", "html.parser"]:
+    for parser_name in ["lxml", ["lxml", "html"], "html5lib", "html.parser"]:
         success = False
         try:
             a = time.time()
-            soup = BeautifulSoup(data, parser)
+            soup = BeautifulSoup(data, parser_name)
             b = time.time()
             success = True
         except Exception as e:
-            print(("%s could not parse the markup." % parser))
+            print(("%s could not parse the markup." % parser_name))
             traceback.print_exc()
         if success:
-            print(("BS4+%s parsed the markup in %.2fs." % (parser, b-a)))
+            print(("BS4+%s parsed the markup in %.2fs." % (parser_name, b-a)))
 
     from lxml import etree
     a = time.time()
@@ -214,7 +226,7 @@ def benchmark_parsers(num_elements=100000):
     b = time.time()
     print(("Raw html5lib parsed the markup in %.2fs." % (b-a)))
 
-def profile(num_elements=100000, parser="lxml"):
+def profile(num_elements:int=100000, parser:str="lxml"):
     """Use Python's profiler on a randomly generated document."""
     filehandle = tempfile.NamedTemporaryFile()
     filename = filehandle.name
diff --git a/bs4/element.py b/bs4/element.py
index 0aefe73..8b3774e 100644
--- a/bs4/element.py
+++ b/bs4/element.py
@@ -1,55 +1,102 @@
+from __future__ import annotations
 # Use of this source code is governed by the MIT license.
 __license__ = "MIT"
 
-try:
-    from collections.abc import Callable # Python 3.6
-except ImportError as e:
-    from collections import Callable
 import re
 import sys
 import warnings
 
 from bs4.css import CSS
+from bs4._deprecation import (
+    _deprecated,
+    _deprecated_alias,
+    _deprecated_function_alias,
+)
 from bs4.formatter import (
     Formatter,
     HTMLFormatter,
     XMLFormatter,
 )
 
-DEFAULT_OUTPUT_ENCODING = "utf-8"
-
-nonwhitespace_re = re.compile(r"\S+")
-
-# NOTE: This isn't used as of 4.7.0. I'm leaving it for a little bit on
-# the off chance someone imported it for their own use.
-whitespace_re = re.compile(r"\s+")
+from typing import (
+    Any,
+    Callable,
+    Dict,
+    Generator,
+    Generic,
+    Iterable,
+    Iterator,
+    List,
+    Mapping,
+    Optional,
+    Pattern,
+    Sequence,
+    Set,
+    TYPE_CHECKING,
+    Tuple,
+    Type,
+    TypeVar,
+    Union,
+    cast,
+)
+from typing_extensions import Self
+if TYPE_CHECKING:
+    from bs4 import BeautifulSoup
+    from bs4.builder import TreeBuilder
+    from bs4.dammit import _Encoding
+    from bs4.formatter import (
+        _EntitySubstitutionFunction,
+        _FormatterOrName,
+    )
+    from bs4._typing import (
+        _AttributeValue,
+        _AttributeValues,
+        _StrainableElement,
+        _StrainableAttribute,
+        _StrainableAttributes,
+        _StrainableString,
+    )
+
+# Deprecated module-level attributes.
+# See https://peps.python.org/pep-0562/
+_deprecated_names = dict(
+    whitespace_re = 'The {name} attribute was deprecated in version 4.7.0. If you need it, make your own copy.'
+)
+#: :meta private:
+_deprecated_whitespace_re: Pattern[str] = re.compile(r"\s+")
 
-def _alias(attr):
-    """Alias one attribute name to another for backward compatibility"""
-    @property
-    def alias(self):
-        return getattr(self, attr)
-
-    @alias.setter
-    def alias(self):
-        return setattr(self, attr)
-    return alias
-
-
-# These encodings are recognized by Python (so PageElement.encode
-# could theoretically support them) but XML and HTML don't recognize
-# them (so they should not show up in an XML or HTML document as that
-# document's encoding).
-#
-# If an XML document is encoded in one of these encodings, no encoding
-# will be mentioned in the XML declaration. If an HTML document is
-# encoded in one of these encodings, and the HTML document has a
-# <meta> tag that mentions an encoding, the encoding will be given as
-# the empty string.
-#
-# Source:
-# https://docs.python.org/3/library/codecs.html#python-specific-encodings
-PYTHON_SPECIFIC_ENCODINGS = set([
+def __getattr__(name):
+    if name in _deprecated_names:
+        message = _deprecated_names[name]
+        warnings.warn(
+            message.format(name=name),
+            DeprecationWarning, stacklevel=2
+        )
+        
+        return globals()[f"_deprecated_{name}"]
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+    
+#: Documents output by Beautiful Soup will be encoded with
+#: this encoding unless you specify otherwise.
+DEFAULT_OUTPUT_ENCODING:str = "utf-8"
+
+#: A regular expression that can be used to split on whitespace.
+nonwhitespace_re: Pattern[str] = re.compile(r"\S+")
+
+#: These encodings are recognized by Python (so `Tag.encode`
+#: could theoretically support them) but XML and HTML don't recognize
+#: them (so they should not show up in an XML or HTML document as that
+#: document's encoding).
+#:
+#: If an XML document is encoded in one of these encodings, no encoding
+#: will be mentioned in the XML declaration. If an HTML document is
+#: encoded in one of these encodings, and the HTML document has a
+#: <meta> tag that mentions an encoding, the encoding will be given as
+#: the empty string.
+#:
+#: Source:
+#: Python documentation, `Python Specific Encodings <https://docs.python.org/3/library/codecs.html#python-specific-encodings>`_
+PYTHON_SPECIFIC_ENCODINGS: Set[_Encoding] = set([
     "idna",
     "mbcs",
     "oem",
@@ -66,11 +113,17 @@ PYTHON_SPECIFIC_ENCODINGS = set([
 
 
 class NamespacedAttribute(str):
-    """A namespaced string (e.g. 'xml:lang') that remembers the namespace
-    ('xml') and the name ('lang') that were used to create it.
+    """A namespaced attribute (e.g. the 'xml:lang' in 'xml:lang="en"')
+    which remembers the namespace prefix ('xml') and the name ('lang')
+    that were used to create it.
     """
 
-    def __new__(cls, prefix, name=None, namespace=None):
+    prefix: Optional[str]
+    name: Optional[str]
+    namespace: Optional[str]
+    
+    def __new__(cls, prefix:Optional[str],
+                name:Optional[str]=None, namespace:Optional[str]=None):
         if not name:
             # This is the default namespace. Its name "has no value"
             # per https://www.w3.org/TR/xml-names/#defaulting
@@ -89,72 +142,126 @@ class NamespacedAttribute(str):
         return obj
 
 class AttributeValueWithCharsetSubstitution(str):
-    """A stand-in object for a character encoding specified in HTML."""
+    """An abstract class standing in for a character encoding specified
+    inside an HTML ``<meta>`` tag.
 
-class CharsetMetaAttributeValue(AttributeValueWithCharsetSubstitution):
-    """A generic stand-in for the value of a meta tag's 'charset' attribute.
+    Subclasses exist for each place such a character encoding might be
+    found: either inside the ``charset`` attribute
+    (`CharsetMetaAttributeValue`) or inside the ``content`` attribute
+    (`ContentMetaAttributeValue`)
 
-    When Beautiful Soup parses the markup '<meta charset="utf8">', the
-    value of the 'charset' attribute will be one of these objects.
+    This allows Beautiful Soup to replace that part of the HTML file
+    with a different encoding when ouputting a tree as a string.
     """
+    # The original, un-encoded value of the ``content`` attribute.
+    #: :meta private:
+    original_value: str
+    
+    def substitute_encoding(self, eventual_encoding:str) -> str:
+        """Do whatever's necessary in this implementation-specific
+        portion an HTML document to substitute in a specific encoding.
+        """
+        raise NotImplementedError()
+
+class CharsetMetaAttributeValue(AttributeValueWithCharsetSubstitution):
+    """A generic stand-in for the value of a ``<meta>`` tag's ``charset``
+    attribute.
+
+    When Beautiful Soup parses the markup ``<meta charset="utf8">``, the
+    value of the ``charset`` attribute will become one of these objects.
 
-    def __new__(cls, original_value):
+    If the document is later encoded to an encoding other than UTF-8, its
+    ``<meta>`` tag will mention the new encoding instead of ``utf8``.
+    """
+    def __new__(cls, original_value:str) -> Self:
+        # We don't need to use the original value for anything, but
+        # it might be useful for the user to know.
         obj = str.__new__(cls, original_value)
         obj.original_value = original_value
         return obj
-
-    def encode(self, encoding):
+    
+    def substitute_encoding(self, eventual_encoding:_Encoding="utf-8") -> str:
         """When an HTML document is being encoded to a given encoding, the
-        value of a meta tag's 'charset' is the name of the encoding.
+        value of a ``<meta>`` tag's ``charset`` becomes the name of
+        the encoding.
         """
-        if encoding in PYTHON_SPECIFIC_ENCODINGS:
+        if eventual_encoding in PYTHON_SPECIFIC_ENCODINGS:
             return ''
-        return encoding
+        return eventual_encoding
 
 
 class ContentMetaAttributeValue(AttributeValueWithCharsetSubstitution):
-    """A generic stand-in for the value of a meta tag's 'content' attribute.
+    """A generic stand-in for the value of a ``<meta>`` tag's ``content``
+    attribute.
 
     When Beautiful Soup parses the markup:
-     <meta http-equiv="content-type" content="text/html; charset=utf8">
+     ``<meta http-equiv="content-type" content="text/html; charset=utf8">``
 
-    The value of the 'content' attribute will be one of these objects.
-    """
-
-    CHARSET_RE = re.compile(r"((^|;)\s*charset=)([^;]*)", re.M)
+    The value of the ``content`` attribute will become one of these objects.
 
-    def __new__(cls, original_value):
+    If the document is later encoded to an encoding other than UTF-8, its
+    ``<meta>`` tag will mention the new encoding instead of ``utf8``.
+    """
+    #: Match the 'charset' argument inside the 'content' attribute
+    #: of a <meta> tag.
+    #: :meta private:
+    CHARSET_RE: Pattern[str] = re.compile(
+        r"((^|;)\s*charset=)([^;]*)", re.M
+    )
+    
+    def __new__(cls, original_value:str) -> Self:
         match = cls.CHARSET_RE.search(original_value)
-        if match is None:
-            # No substitution necessary.
-            return str.__new__(str, original_value)
-
         obj = str.__new__(cls, original_value)
         obj.original_value = original_value
         return obj
 
-    def encode(self, encoding):
-        if encoding in PYTHON_SPECIFIC_ENCODINGS:
-            return ''
+    def substitute_encoding(self, eventual_encoding:_Encoding="utf-8") -> str:
+        """When an HTML document is being encoded to a given encoding, the
+        value of the ``charset=`` in a ``<meta>`` tag's ``content`` becomes
+        the name of the encoding.
+        """
+        if eventual_encoding in PYTHON_SPECIFIC_ENCODINGS:
+            return self.CHARSET_RE.sub('', self.original_value)
         def rewrite(match):
-            return match.group(1) + encoding
+            return match.group(1) + eventual_encoding
         return self.CHARSET_RE.sub(rewrite, self.original_value)
 
 
 class PageElement(object):
-    """Contains the navigational information for some part of the page:
-    that is, its current location in the parse tree.
+    """An abstract class representing a single element in the parse tree.
 
-    NavigableString, Tag, etc. are all subclasses of PageElement.
+    `NavigableString`, `Tag`, etc. are all subclasses of
+    `PageElement`. For this reason you'll see a lot of methods that
+    return `PageElement`, but you'll never see an actual `PageElement`
+    object. For the most part you can think of `PageElement` as
+    meaning "a `Tag` or a `NavigableString`."
     """
 
-    # In general, we can't tell just by looking at an element whether
-    # it's contained in an XML document or an HTML document. But for
-    # Tags (q.v.) we can store this information at parse time.
-    known_xml = None
-
-    def setup(self, parent=None, previous_element=None, next_element=None,
-              previous_sibling=None, next_sibling=None):
+    #: In general, we can't tell just by looking at an element whether
+    #: it's contained in an XML document or an HTML document. But for
+    #: `Tag` objects (q.v.) we can store this information at parse time.
+    #: :meta private:
+    known_xml: Optional[bool] = None
+
+    #: Whether or not this element has been decomposed from the tree
+    #: it was created in.
+    _decomposed: bool
+
+    parent: Optional[Tag]
+    next_element: Optional[PageElement]
+    previous_element: Optional[PageElement]
+    next_sibling: Optional[PageElement]
+    previous_sibling: Optional[PageElement]
+
+    #: Whether or not this element is hidden from generated output.
+    #: Only the `BeautifulSoup` object itself is hidden.
+    hidden: bool=False
+    
+    def setup(self, parent:Optional[Tag]=None,
+              previous_element:Optional[PageElement]=None,
+              next_element:Optional[PageElement]=None,
+              previous_sibling:Optional[PageElement]=None,
+              next_sibling:Optional[PageElement]=None) -> None:
         """Sets up the initial relations between this element and
         other elements.
 
@@ -175,7 +282,7 @@ class PageElement(object):
         self.parent = parent
 
         self.previous_element = previous_element
-        if previous_element is not None:
+        if self.previous_element is not None:
             self.previous_element.next_element = self
 
         self.next_element = next_element
@@ -191,10 +298,10 @@ class PageElement(object):
             previous_sibling = self.parent.contents[-1]
 
         self.previous_sibling = previous_sibling
-        if previous_sibling is not None:
+        if self.previous_sibling is not None:
             self.previous_sibling.next_sibling = self
 
-    def format_string(self, s, formatter):
+    def format_string(self, s:str, formatter:Optional[_FormatterOrName]) -> str:
         """Format the given string using the given formatter.
 
         :param s: A string.
@@ -207,28 +314,35 @@ class PageElement(object):
         output = formatter.substitute(s)
         return output
 
-    def formatter_for_name(self, formatter):
+    def formatter_for_name(
+        self,
+        formatter_name:Union[_FormatterOrName, _EntitySubstitutionFunction]
+    ) -> Formatter:
         """Look up or create a Formatter for the given identifier,
         if necessary.
 
-        :param formatter: Can be a Formatter object (used as-is), a
+        :param formatter: Can be a `Formatter` object (used as-is), a
             function (used as the entity substitution hook for an
-            XMLFormatter or HTMLFormatter), or a string (used to look
-            up an XMLFormatter or HTMLFormatter in the appropriate
+            `XMLFormatter` or `HTMLFormatter`), or a string (used to look
+            up an `XMLFormatter` or `HTMLFormatter` in the appropriate
             registry.
         """
-        if isinstance(formatter, Formatter):
-            return formatter
+        if isinstance(formatter_name, Formatter):
+            return formatter_name
+        c: type[Formatter]
+        registry: Mapping[Optional[str], Formatter]
         if self._is_xml:
             c = XMLFormatter
+            registry = XMLFormatter.REGISTRY
         else:
             c = HTMLFormatter
-        if isinstance(formatter, Callable):
-            return c(entity_substitution=formatter)
-        return c.REGISTRY[formatter]
+            registry = HTMLFormatter.REGISTRY
+        if callable(formatter_name):
+            return c(entity_substitution=formatter_name)
+        return registry[formatter_name]
 
     @property
-    def _is_xml(self):
+    def _is_xml(self) -> bool:
         """Is this element part of an XML tree or an HTML tree?
 
         This is used in formatter_for_name, when deciding whether an
@@ -250,28 +364,41 @@ class PageElement(object):
             return getattr(self, 'is_xml', False)
         return self.parent._is_xml
 
-    nextSibling = _alias("next_sibling")  # BS3
-    previousSibling = _alias("previous_sibling")  # BS3
+    nextSibling = _deprecated_alias("nextSibling", "next_sibling", "4.0.0")
+    previousSibling = _deprecated_alias(
+        "previousSibling", "previous_sibling", "4.0.0"
+    )
 
-    default = object()
-    def _all_strings(self, strip=False, types=default):
+    def __deepcopy__(self, memo:Dict, recursive:bool=False) -> Self:
+        raise NotImplementedError()
+        
+    def __copy__(self) -> Self:
+        """A copy of a PageElement can only be a deep copy, because
+        only one PageElement can occupy a given place in a parse tree.
+        """
+        return self.__deepcopy__({})
+    
+    default: Iterable[type[NavigableString]] = tuple() #: :meta private:
+    def _all_strings(self, strip:bool=False, types:Iterable[type[NavigableString]]=default) -> Iterator[str]:
         """Yield all strings of certain classes, possibly stripping them.
 
-        This is implemented differently in Tag and NavigableString.
+        This is implemented differently in `Tag` and `NavigableString`.
         """
         raise NotImplementedError()
 
     @property
-    def stripped_strings(self):
-        """Yield all strings in this PageElement, stripping them first.
+    def stripped_strings(self) -> Iterator[str]:
+        """Yield all interesting strings in this PageElement, stripping them
+        first.
 
-        :yield: A sequence of stripped strings.
+        See `Tag` for information on which strings are considered
+        interesting in a given context.
         """
         for string in self._all_strings(True):
             yield string
 
-    def get_text(self, separator="", strip=False,
-                 types=default):
+    def get_text(self, separator:str="", strip:bool=False,
+                 types:Iterable[Type[NavigableString]]=default) -> str:
         """Get all child strings of this PageElement, concatenated using the
         given separator.
 
@@ -294,19 +421,19 @@ class PageElement(object):
     getText = get_text
     text = property(get_text)
 
-    def replace_with(self, *args):
-        """Replace this PageElement with one or more PageElements, keeping the
-        rest of the tree the same.
+    def replace_with(self, *args:PageElement) -> PageElement:
+        """Replace this `PageElement` with one or more other `PageElements`,
+        keeping the rest of the tree the same.
 
-        :param args: One or more PageElements.
-        :return: `self`, no longer part of the tree.
+        :return: This `PageElement`, no longer part of the tree.
         """
         if self.parent is None:
             raise ValueError(
                 "Cannot replace one element with another when the "
                 "element to be replaced is not part of a tree.")
         if len(args) == 1 and args[0] is self:
-            return
+            # Replacing an element with itself is a no-op.
+            return self
         if any(x is self.parent for x in args):
             raise ValueError("Cannot replace a Tag with its parent.")
         old_parent = self.parent
@@ -315,45 +442,28 @@ class PageElement(object):
         for idx, replace_with in enumerate(args, start=my_index):
             old_parent.insert(idx, replace_with)
         return self
-    replaceWith = replace_with  # BS3
+    replaceWith = _deprecated_function_alias(
+        "replaceWith", "replace_with", "4.0.0"
+    )
 
-    def unwrap(self):
-        """Replace this PageElement with its contents.
+    def wrap(self, wrap_inside:Tag) -> Tag:
+        """Wrap this `PageElement` inside a `Tag`.
 
-        :return: `self`, no longer part of the tree.
-        """
-        my_parent = self.parent
-        if self.parent is None:
-            raise ValueError(
-                "Cannot replace an element with its contents when that"
-                "element is not part of a tree.")
-        my_index = self.parent.index(self)
-        self.extract(_self_index=my_index)
-        for child in reversed(self.contents[:]):
-            my_parent.insert(my_index, child)
-        return self
-    replace_with_children = unwrap
-    replaceWithChildren = unwrap  # BS3
-
-    def wrap(self, wrap_inside):
-        """Wrap this PageElement inside another one.
-
-        :param wrap_inside: A PageElement.
-        :return: `wrap_inside`, occupying the position in the tree that used
-           to be occupied by `self`, and with `self` inside it.
+        :return: ``wrap_inside``, occupying the position in the tree that used
+           to be occupied by this object, and with this object now inside it.
         """
         me = self.replace_with(wrap_inside)
         wrap_inside.append(me)
         return wrap_inside
 
-    def extract(self, _self_index=None):
+    def extract(self, _self_index:Optional[int]=None) -> PageElement:
         """Destructively rips this element out of the tree.
 
         :param _self_index: The location of this element in its parent's
            .contents, if known. Passing this in allows for a performance
            optimization.
 
-        :return: `self`, no longer part of the tree.
+        :return: this `PageElement`, no longer part of the tree.
         """
         if self.parent is not None:
             if _self_index is None:
@@ -364,11 +474,17 @@ class PageElement(object):
         #this element (and any children) hadn't been parsed. Connect
         #the two.
         last_child = self._last_descendant()
+
+        # last_child can't be None because we passed accept_self=True
+        # into _last_descendant. Worst case, last_child will be
+        # self. Making this cast removes several mypy complaints later
+        # on as we manipulate last_child.
+        last_child = cast(PageElement, last_child)
         next_element = last_child.next_element
 
-        if (self.previous_element is not None and
-            self.previous_element is not next_element):
-            self.previous_element.next_element = next_element
+        if self.previous_element is not None:
+            if self.previous_element is not next_element:
+                self.previous_element.next_element = next_element
         if next_element is not None and next_element is not self.previous_element:
             next_element.previous_element = self.previous_element
         self.previous_element = None
@@ -384,12 +500,38 @@ class PageElement(object):
         self.previous_sibling = self.next_sibling = None
         return self
 
-    def _last_descendant(self, is_initialized=True, accept_self=True):
+    def decompose(self) -> None:
+        """Recursively destroys this `PageElement` and its children.
+
+        The element will be removed from the tree and wiped out; so
+        will everything beneath it.
+
+        The behavior of a decomposed `PageElement` is undefined and you
+        should never use one for anything, but if you need to *check*
+        whether an element has been decomposed, you can use the
+        `PageElement.decomposed` property.
+        """
+        self.extract()
+        e: Optional[PageElement] = self
+        next_up: Optional[PageElement] = None
+        while e is not None:
+            next_up = e.next_element
+            e.__dict__.clear()
+            if isinstance(e, Tag):
+                e.contents = []
+            e._decomposed = True
+            e = next_up
+
+    def _last_descendant(
+        self, is_initialized:bool=True, accept_self:bool=True
+    ) -> Optional[PageElement]:
         """Finds the last element beneath this object to be parsed.
 
-        :param is_initialized: Has `setup` been called on this PageElement
-            yet?
-        :param accept_self: Is `self` an acceptable answer to the question?
+        :param is_initialized: Has `PageElement.setup` been called on
+            this `PageElement` yet?
+
+        :param accept_self: Is ``self`` an acceptable answer to the
+            question?
         """
         if is_initialized and self.next_sibling is not None:
             last_child = self.next_sibling.previous_element
@@ -400,121 +542,15 @@ class PageElement(object):
         if not accept_self and last_child is self:
             last_child = None
         return last_child
-    # BS3: Not part of the API!
-    _lastRecursiveChild = _last_descendant
 
-    def insert(self, position, new_child):
-        """Insert a new PageElement in the list of this PageElement's children.
-
-        This works the same way as `list.insert`.
-
-        :param position: The numeric position that should be occupied
-           in `self.children` by the new PageElement.
-        :param new_child: A PageElement.
-        """
-        if new_child is None:
-            raise ValueError("Cannot insert None into a tag.")
-        if new_child is self:
-            raise ValueError("Cannot insert a tag into itself.")
-        if (isinstance(new_child, str)
-            and not isinstance(new_child, NavigableString)):
-            new_child = NavigableString(new_child)
+    _lastRecursiveChild = _deprecated_alias("_lastRecursiveChild", "_last_descendant", "4.0.0")
 
-        from bs4 import BeautifulSoup
-        if isinstance(new_child, BeautifulSoup):
-            # We don't want to end up with a situation where one BeautifulSoup
-            # object contains another. Insert the children one at a time.
-            for subchild in list(new_child.contents):
-                self.insert(position, subchild)
-                position += 1
-            return
-        position = min(position, len(self.contents))
-        if hasattr(new_child, 'parent') and new_child.parent is not None:
-            # We're 'inserting' an element that's already one
-            # of this object's children.
-            if new_child.parent is self:
-                current_index = self.index(new_child)
-                if current_index < position:
-                    # We're moving this element further down the list
-                    # of this object's children. That means that when
-                    # we extract this element, our target index will
-                    # jump down one.
-                    position -= 1
-            new_child.extract()
-
-        new_child.parent = self
-        previous_child = None
-        if position == 0:
-            new_child.previous_sibling = None
-            new_child.previous_element = self
-        else:
-            previous_child = self.contents[position - 1]
-            new_child.previous_sibling = previous_child
-            new_child.previous_sibling.next_sibling = new_child
-            new_child.previous_element = previous_child._last_descendant(False)
-        if new_child.previous_element is not None:
-            new_child.previous_element.next_element = new_child
-
-        new_childs_last_element = new_child._last_descendant(False)
-
-        if position >= len(self.contents):
-            new_child.next_sibling = None
-
-            parent = self
-            parents_next_sibling = None
-            while parents_next_sibling is None and parent is not None:
-                parents_next_sibling = parent.next_sibling
-                parent = parent.parent
-                if parents_next_sibling is not None:
-                    # We found the element that comes next in the document.
-                    break
-            if parents_next_sibling is not None:
-                new_childs_last_element.next_element = parents_next_sibling
-            else:
-                # The last element of this tag is the last element in
-                # the document.
-                new_childs_last_element.next_element = None
-        else:
-            next_child = self.contents[position]
-            new_child.next_sibling = next_child
-            if new_child.next_sibling is not None:
-                new_child.next_sibling.previous_sibling = new_child
-            new_childs_last_element.next_element = next_child
-
-        if new_childs_last_element.next_element is not None:
-            new_childs_last_element.next_element.previous_element = new_childs_last_element
-        self.contents.insert(position, new_child)
-
-    def append(self, tag):
-        """Appends the given PageElement to the contents of this one.
-
-        :param tag: A PageElement.
-        """
-        self.insert(len(self.contents), tag)
-
-    def extend(self, tags):
-        """Appends the given PageElements to this one's contents.
-
-        :param tags: A list of PageElements. If a single Tag is
-            provided instead, this PageElement's contents will be extended
-            with that Tag's contents.
-        """
-        if isinstance(tags, Tag):
-            tags = tags.contents
-        if isinstance(tags, list):
-            # Moving items around the tree may change their position in
-            # the original list. Make a list that won't change.
-            tags = list(tags)
-        for tag in tags:
-            self.append(tag)
-
-    def insert_before(self, *args):
+    def insert_before(self, *args:PageElement) -> None:
         """Makes the given element(s) the immediate predecessor of this one.
 
-        All the elements will have the same parent, and the given elements
-        will be immediately before this one.
-
-        :param args: One or more PageElements.
+        All the elements will have the same `PageElement.parent` as
+        this one, and the given elements will occur immediately before
+        this one.
         """
         parent = self.parent
         if parent is None:
@@ -530,13 +566,12 @@ class PageElement(object):
             index = parent.index(self)
             parent.insert(index, predecessor)
 
-    def insert_after(self, *args):
+    def insert_after(self, *args:PageElement) -> None:
         """Makes the given element(s) the immediate successor of this one.
 
-        The elements will have the same parent, and the given elements
-        will be immediately after this one.
-
-        :param args: One or more PageElements.
+        The elements will have the same `PageElement.parent` as this
+        one, and the given elements will occur immediately after this
+        one.
         """
         # Do all error checking before modifying the tree.
         parent = self.parent
@@ -556,7 +591,14 @@ class PageElement(object):
             parent.insert(index+1+offset, successor)
             offset += 1
 
-    def find_next(self, name=None, attrs={}, string=None, **kwargs):
+    def find_next(
+            self,
+            name:Optional[_StrainableElement]=None,
+            attrs:_StrainableAttributes={},
+            string:Optional[_StrainableString]=None,
+            node:Optional[_TagOrStringMatchFunction]=None,
+            **kwargs:_StrainableAttribute
+    ) -> Optional[PageElement]:
         """Find the first PageElement that matches the given criteria and
         appears later in the document than this PageElement.
 
@@ -564,36 +606,47 @@ class PageElement(object):
         documentation for detailed explanations.
 
         :param name: A filter on tag name.
-        :param attrs: A dictionary of filters on attribute values.
+        :param attrs: Additional filters on attribute values.
         :param string: A filter for a NavigableString with specific text.
-        :kwargs: A dictionary of filters on attribute values.
-        :return: A PageElement.
-        :rtype: bs4.element.Tag | bs4.element.NavigableString
-        """
-        return self._find_one(self.find_all_next, name, attrs, string, **kwargs)
-    findNext = find_next  # BS3
-
-    def find_all_next(self, name=None, attrs={}, string=None, limit=None,
-                    **kwargs):
-        """Find all PageElements that match the given criteria and appear
-        later in the document than this PageElement.
+        :kwargs: Additional filters on attribute values.
+        """
+        return self._find_one(self.find_all_next, name, attrs, string, node, **kwargs)
+    findNext = _deprecated_function_alias("findNext", "find_next", "4.0.0")
+
+    def find_all_next(
+            self,
+            name:Optional[_StrainableElement]=None,
+            attrs:_StrainableAttributes={},
+            string:Optional[_StrainableString]=None,
+            limit:Optional[int]=None,
+            node:Optional[_TagOrStringMatchFunction]=None,
+            _stacklevel:int=2,
+            **kwargs:_StrainableAttribute
+    ) -> ResultSet[PageElement]: 
+        """Find all `PageElement` objects that match the given criteria and
+        appear later in the document than this `PageElement`.
 
         All find_* methods take a common set of arguments. See the online
         documentation for detailed explanations.
 
         :param name: A filter on tag name.
-        :param attrs: A dictionary of filters on attribute values.
+        :param attrs: Additional filters on attribute values.
         :param string: A filter for a NavigableString with specific text.
         :param limit: Stop looking after finding this many results.
-        :kwargs: A dictionary of filters on attribute values.
-        :return: A ResultSet containing PageElements.
+        :param _stacklevel: Used internally to improve warning messages.
+        :kwargs: Additional filters on attribute values.
         """
-        _stacklevel = kwargs.pop('_stacklevel', 2)
         return self._find_all(name, attrs, string, limit, self.next_elements,
-                              _stacklevel=_stacklevel+1, **kwargs)
-    findAllNext = find_all_next  # BS3
-
-    def find_next_sibling(self, name=None, attrs={}, string=None, **kwargs):
+                              node, _stacklevel=_stacklevel+1, **kwargs)
+    findAllNext = _deprecated_function_alias("findAllNext", "find_all_next", "4.0.0")
+
+    def find_next_sibling(
+            self,
+            name:Optional[_StrainableElement]=None,
+            attrs:_StrainableAttributes={},
+            string:Optional[_StrainableString]=None,
+            node:Optional[_TagOrStringMatchFunction]=None,
+            **kwargs:_StrainableAttribute) -> Optional[PageElement]:
         """Find the closest sibling to this PageElement that matches the
         given criteria and appears later in the document.
 
@@ -601,102 +654,143 @@ class PageElement(object):
         online documentation for detailed explanations.
 
         :param name: A filter on tag name.
-        :param attrs: A dictionary of filters on attribute values.
-        :param string: A filter for a NavigableString with specific text.
-        :kwargs: A dictionary of filters on attribute values.
-        :return: A PageElement.
-        :rtype: bs4.element.Tag | bs4.element.NavigableString
+        :param attrs: Additional filters on attribute values.
+        :param string: A filter for a `NavigableString` with specific text.
+        :kwargs: Additional filters on attribute values.
         """
         return self._find_one(self.find_next_siblings, name, attrs, string,
-                             **kwargs)
-    findNextSibling = find_next_sibling  # BS3
-
-    def find_next_siblings(self, name=None, attrs={}, string=None, limit=None,
-                           **kwargs):
-        """Find all siblings of this PageElement that match the given criteria
+                              node, **kwargs)
+    findNextSibling = _deprecated_function_alias(
+        "findNextSibling", "find_next_sibling", "4.0.0"
+    )
+
+    def find_next_siblings(
+            self,
+            name:Optional[_StrainableElement]=None,
+            attrs:_StrainableAttributes={},
+            string:Optional[_StrainableString]=None,
+            limit:Optional[int]=None,
+            node:Optional[_TagOrStringMatchFunction]=None,
+            _stacklevel:int=2,
+            **kwargs:_StrainableAttribute
+    ) -> ResultSet[PageElement]: 
+        """Find all siblings of this `PageElement` that match the given criteria
         and appear later in the document.
 
         All find_* methods take a common set of arguments. See the online
         documentation for detailed explanations.
 
         :param name: A filter on tag name.
-        :param attrs: A dictionary of filters on attribute values.
-        :param string: A filter for a NavigableString with specific text.
+        :param attrs: Additional filters on attribute values.
+        :param string: A filter for a `NavigableString` with specific text.
         :param limit: Stop looking after finding this many results.
-        :kwargs: A dictionary of filters on attribute values.
-        :return: A ResultSet of PageElements.
-        :rtype: bs4.element.ResultSet
+        :param _stacklevel: Used internally to improve warning messages.
+        :kwargs: Additional filters on attribute values.
         """
-        _stacklevel = kwargs.pop('_stacklevel', 2)
         return self._find_all(
             name, attrs, string, limit,
-            self.next_siblings, _stacklevel=_stacklevel+1, **kwargs
+            self.next_siblings, node, _stacklevel=_stacklevel+1, **kwargs
         )
-    findNextSiblings = find_next_siblings   # BS3
-    fetchNextSiblings = find_next_siblings  # BS2
-
-    def find_previous(self, name=None, attrs={}, string=None, **kwargs):
-        """Look backwards in the document from this PageElement and find the
-        first PageElement that matches the given criteria.
+    findNextSiblings = _deprecated_function_alias(
+        "findNextSiblings", "find_next_siblings", "4.0.0"
+    )
+    fetchNextSiblings = _deprecated_function_alias(
+        "fetchNextSiblings", "find_next_siblings", "3.0.0"
+    )
+
+    def find_previous(
+            self,
+            name:Optional[_StrainableElement]=None,
+            attrs:_StrainableAttributes={},
+            string:Optional[_StrainableString]=None,
+            node:Optional[_TagOrStringMatchFunction]=None,
+            **kwargs:_StrainableAttribute) -> Optional[PageElement]:
+        """Look backwards in the document from this `PageElement` and find the
+        first `PageElement` that matches the given criteria.
 
         All find_* methods take a common set of arguments. See the online
         documentation for detailed explanations.
 
         :param name: A filter on tag name.
-        :param attrs: A dictionary of filters on attribute values.
-        :param string: A filter for a NavigableString with specific text.
-        :kwargs: A dictionary of filters on attribute values.
-        :return: A PageElement.
-        :rtype: bs4.element.Tag | bs4.element.NavigableString
+        :param attrs: Additional filters on attribute values.
+        :param string: A filter for a `NavigableString` with specific text.
+        :kwargs: Additional filters on attribute values.
         """
         return self._find_one(
-            self.find_all_previous, name, attrs, string, **kwargs)
-    findPrevious = find_previous  # BS3
-
-    def find_all_previous(self, name=None, attrs={}, string=None, limit=None,
-                        **kwargs):
-        """Look backwards in the document from this PageElement and find all
-        PageElements that match the given criteria.
+            self.find_all_previous, name, attrs, string, node, **kwargs)
+
+    findPrevious = _deprecated_function_alias(
+        "findPrevious", "find_previous", "3.0.0"
+    )
+
+    def find_all_previous(
+            self,
+            name:Optional[_StrainableElement]=None,
+            attrs:_StrainableAttributes={},
+            string:Optional[_StrainableString]=None,
+            limit:Optional[int]=None,
+            node:Optional[_TagOrStringMatchFunction]=None,
+            _stacklevel:int=2,
+            **kwargs:_StrainableAttribute
+    ) -> ResultSet[PageElement]: 
+        """Look backwards in the document from this `PageElement` and find all
+        `PageElement` that match the given criteria.
 
         All find_* methods take a common set of arguments. See the online
         documentation for detailed explanations.
 
         :param name: A filter on tag name.
-        :param attrs: A dictionary of filters on attribute values.
-        :param string: A filter for a NavigableString with specific text.
+        :param attrs: Additional filters on attribute values.
+        :param string: A filter for a `NavigableString` with specific text.
         :param limit: Stop looking after finding this many results.
-        :kwargs: A dictionary of filters on attribute values.
-        :return: A ResultSet of PageElements.
-        :rtype: bs4.element.ResultSet
+        :param _stacklevel: Used internally to improve warning messages.
+        :kwargs: Additional filters on attribute values.
         """
-        _stacklevel = kwargs.pop('_stacklevel', 2)
         return self._find_all(
             name, attrs, string, limit, self.previous_elements,
-            _stacklevel=_stacklevel+1, **kwargs
+            node, _stacklevel=_stacklevel+1, **kwargs
         )
-    findAllPrevious = find_all_previous  # BS3
-    fetchPrevious = find_all_previous    # BS2
-
-    def find_previous_sibling(self, name=None, attrs={}, string=None, **kwargs):
-        """Returns the closest sibling to this PageElement that matches the
+    findAllPrevious = _deprecated_function_alias(
+        "findAllPrevious", "find_all_previous", "4.0.0"
+    )
+    fetchAllPrevious = _deprecated_function_alias(
+        "fetchAllPrevious", "find_all_previous", "3.0.0"
+    )
+
+    def find_previous_sibling(
+            self,
+            name:Optional[_StrainableElement]=None,
+            attrs:_StrainableAttributes={},
+            string:Optional[_StrainableString]=None,
+            node:Optional[_TagOrStringMatchFunction]=None,
+            **kwargs:_StrainableAttribute) -> Optional[PageElement]:
+        """Returns the closest sibling to this `PageElement` that matches the
         given criteria and appears earlier in the document.
 
         All find_* methods take a common set of arguments. See the online
         documentation for detailed explanations.
 
         :param name: A filter on tag name.
-        :param attrs: A dictionary of filters on attribute values.
-        :param string: A filter for a NavigableString with specific text.
-        :kwargs: A dictionary of filters on attribute values.