{"id":1115,"date":"2025-11-30T23:02:46","date_gmt":"2025-11-30T14:02:46","guid":{"rendered":"https:\/\/www.mindware-jp.com\/en\/?p=1115"},"modified":"2025-12-01T09:40:01","modified_gmt":"2025-12-01T00:40:01","slug":"data-science-and-buddhism-from-the-ugly-duckling-theorem-to-emptiness-provisionality-and-the-middle-way","status":"publish","type":"post","link":"https:\/\/www.mindware-jp.com\/en\/2025\/11\/30\/data-science-and-buddhism-from-the-ugly-duckling-theorem-to-emptiness-provisionality-and-the-middle-way\/","title":{"rendered":"Data Science and Buddhism: From the \u201cUgly Duckling Theorem\u201d to Emptiness, Provisionality, and the Middle Way"},"content":{"rendered":"\n

The pioneering research that shaped today\u2019s \u201cdata science\u201d can be traced back to pattern recognition studies of the 1960s. In Japan, that era saw the development of machines capable of reading handwritten postal codes to sort mail at high speed. One of the intellectual leaders of that formative period, Satoshi Watanabe<\/strong>, proposed the remarkably evocative concept known as the \u201cUgly Duckling Theorem.\u201d<\/strong><\/p>\n\n\n\n

Despite its importance, this theorem is seldom taught in modern data science programs, and many data scientists\u2014especially those trained within frameworks dominated by machine learning engineering\u2014have never seriously engaged with its philosophical implications. Yet, once examined carefully, the theorem reveals profound connections to Buddhist philosophy, Kantian epistemology, Husserl\u2019s phenomenology, and Jakob von Uexk\u00fcll\u2019s concept of Umwelt (environmental world).<\/p>\n\n\n\n

This essay explores those connections and argues that a proper understanding of clustering requires the same cognitive attitude Buddhism describes as the Middle Way<\/strong> between emptiness and conventional appearance.<\/p>\n\n\n\n

\n\n\n\n

\u25a0 What Is the Ugly Duckling Theorem?<\/h2>\n\n\n\n

Watanabe famously stated:<\/p>\n\n\n\n

\n

If we measure similarity by the number of predicates they share, then any two objects possess the same degree of similarity.<\/strong><\/p>\n<\/blockquote>\n\n\n\n

This sounds counterintuitive at first, but the logic is straightforward:<\/p>\n\n\n\n

    \n
  • The world contains an unlimited number of possible attributes.<\/li>\n\n\n\n
  • Humans select only a tiny subset when comparing objects.<\/li>\n\n\n\n
  • Any two objects share some attributes and differ in others.<\/li>\n\n\n\n
  • Without assigning importance<\/strong> (weights) to attributes, all pairs of objects are equally similar.<\/li>\n<\/ul>\n\n\n\n

    Therefore, similarity is not an inherent property of the data<\/strong>.
    It is a consequence of which attributes we choose to value<\/strong>.<\/p>\n\n\n\n

    Watanabe concluded:<\/p>\n\n\n\n

    \n

    \u201cTo escape the Ugly Duckling Theorem, one must admit that some attributes are more important than others.\u201d<\/strong>
    \u201cThe goal of clustering is not to discover \u2018objective\u2019 classes but to create useful<\/em> new classifications.\u201d<\/strong><\/p>\n<\/blockquote>\n\n\n\n

    However, many practitioners overlook this foundational point. They assume, often unconsciously, that:<\/p>\n\n\n\n

      \n
    • A \u201ctrue\u201d structure exists inside the data.<\/li>\n\n\n\n
    • A clustering algorithm can reveal it.<\/li>\n\n\n\n
    • The result should coincide with everyday categories.<\/li>\n\n\n\n
    • The clustering with the highest quality metric (e.g., silhouette score) must be the \u201ccorrect\u201d one.<\/li>\n<\/ul>\n\n\n\n

      Clustering metrics are indeed helpful, but treating them as arbiters of objective reality is na\u00efve and even dangerous. They are evaluative tools\u2014not detectors of ontological truth<\/strong>.<\/p>\n\n\n\n

      \n\n\n\n

      \u25a0 Philosophical Depth: Emptiness, the Thing-in-Itself, Phenomenology, and Umwelt<\/h2>\n\n\n\n

      The Ugly Duckling Theorem resonates with several major philosophical traditions.<\/p>\n\n\n\n

      \u25cf Buddhism\u2019s Doctrine of Emptiness (\u015a\u016bnyat\u0101)<\/h3>\n\n\n\n
        \n
      • No object possesses inherent essence or fixed identity.<\/li>\n\n\n\n
      • Categories, meanings, and distinctions arise only through human conceptual activity.<\/li>\n\n\n\n
      • Classification is therefore empty<\/em>: it does not mirror an objective essence.<\/li>\n<\/ul>\n\n\n\n

        \u25cf Kant\u2019s \u201cThing-in-Itself\u201d<\/h3>\n\n\n\n
          \n
        • Humans can only perceive the world through the structure of human sensibility (space, time, categories).<\/li>\n\n\n\n
        • The world \u201cas it is\u201d has no color, smell, value, or meaning independent of perception.<\/li>\n<\/ul>\n\n\n\n

          \u25cf Husserl\u2019s Phenomenology and \u201cEpoch\u00e9\u201d<\/h3>\n\n\n\n
            \n
          • Suspend judgments and preconceived categories.<\/li>\n\n\n\n
          • Attend to phenomena as they appear, without the distortion of habitual interpretation.<\/li>\n<\/ul>\n\n\n\n

            \u25cf Uexk\u00fcll\u2019s Umwelt<\/h3>\n\n\n\n
              \n
            • Every organism inhabits a world structured by its sensory and bodily constitution.<\/li>\n\n\n\n
            • Each species lives in its own \u201cmeaningful world.\u201d<\/li>\n<\/ul>\n\n\n\n

              Humans, too, live within a human-shaped Umwelt.
              Classification cannot be objective because the world we see is already filtered through human embodiment and cognition.<\/strong><\/p>\n\n\n\n

              \n\n\n\n

              \u25a0 The Buddhist Framework of \u201cThree Truths\u201d: Emptiness, Provisionality, and the Middle<\/h2>\n\n\n\n

              The Tiantai (Tendai) philosopher Zhiyi<\/strong> articulated a threefold truth that maps surprisingly well onto the epistemology of data analysis:<\/p>\n\n\n\n

              1. Emptiness (k\u016b-tai):<\/strong><\/h3>\n\n\n\n

              All phenomena lack inherent essence; classifications do not exist \u201cin themselves.\u201d<\/p>\n\n\n\n

              In data terms:
              The raw data contains no labels, no clusters, no \u201cnatural kinds.\u201d<\/strong><\/p>\n\n\n\n

              2. Provisional Existence (ke-tai):<\/strong><\/h3>\n\n\n\n

              Phenomena appear as meaningful, labeled, and structured from a human standpoint<\/strong>.<\/p>\n\n\n\n

              In data terms:
              Human goals, tasks, and interpretations create useful classifications<\/strong> for reasoning, communication, and decision-making.<\/p>\n\n\n\n

              Clustering belongs entirely to this realm of provisionality<\/em>.<\/p>\n\n\n\n

              3. The Middle (ch\u016b-tai):<\/strong><\/h3>\n\n\n\n

              A non-dual perspective that embraces both emptiness and provisionality without clinging to either.<\/p>\n\n\n\n

              This corresponds to the phenomenological epoch\u00e9<\/strong>\u2014neither assuming inherent structure (emptiness), nor denying the usefulness of structure (provisionality).<\/p>\n\n\n\n

              Applied to data science, this is the attitude we must cultivate when working with clustering.<\/p>\n\n\n\n

              \n\n\n\n

              \u25a0 The Practical Analogy: Drawing, Pattern Perception, and the Artist\u2019s Eye<\/h2>\n\n\n\n

              Epoch\u00e9 is notoriously difficult to practice.
              A helpful analogy comes from the art of drawing.<\/p>\n\n\n\n

                \n
              • Novice artists draw \u201ca nose\u201d or \u201can apple,\u201d guided by conceptual preconceptions.<\/li>\n\n\n\n
              • Skilled artists temporarily set aside the object\u2019s identity and observe pure shapes, values, proportions, edges<\/strong>.<\/li>\n<\/ul>\n\n\n\n

                This is an act of suspending conceptual thought\u2014an epoch\u00e9.
                In drawing, auxiliary lines<\/strong> help reveal structure:<\/p>\n\n\n\n

                  \n
                • These lines do not exist in reality.<\/li>\n\n\n\n
                • Yet drawing them aids understanding and skillful depiction.<\/li>\n<\/ul>\n\n\n\n

                  Clustering serves the same function:<\/p>\n\n\n\n

                  \n

                  A cluster is not a \u201creal boundary\u201d in the data.
                  It is an auxiliary line that helps us understand patterns.<\/strong><\/p>\n<\/blockquote>\n\n\n\n

                  Mistaking auxiliary lines for objective boundaries is the central error the Ugly Duckling Theorem warns against.<\/p>\n\n\n\n

                  \n\n\n\n

                  \u25a0 Conclusion: The Middle Way Is the Secret to Using Clustering Wisely<\/h2>\n\n\n\n

                  Clustering is powerful precisely because:<\/p>\n\n\n\n

                    \n
                  • It does not<\/strong> discover immutable categories.<\/li>\n\n\n\n
                  • It creates<\/strong> classifications that are useful for a purpose.<\/li>\n\n\n\n
                  • It reflects the analyst\u2019s decisions, values, and chosen attributes.<\/li>\n<\/ul>\n\n\n\n

                    Thus the true mastery of clustering lies in:<\/p>\n\n\n\n

                      \n
                    • Emptiness:<\/strong> Recognizing that classifications are not inherently real.<\/li>\n\n\n\n
                    • Provisionality:<\/strong> Using classifications as tools for understanding and action.<\/li>\n\n\n\n
                    • The Middle:<\/strong> Maintaining flexibility, detachment, and methodological humility.<\/li>\n<\/ul>\n\n\n\n

                      This Middle Way is not only a Buddhist teaching but also the cognitive stance required for sophisticated data analysis.<\/p>\n\n\n\n

                      The Ugly Duckling Theorem invites us to see the limits of our models.
                      The Buddhist three truths teach us how to work productively within those limits.
                      Together, they form a philosophical foundation for a wiser, more reflective data science.<\/p>\n","protected":false},"excerpt":{"rendered":"

                      The pioneering research that shaped today\u2019s \u201cdata science\u201d can be traced back to pattern recognition studies of the 1960s. In Japan, that era saw the development of machines capable of reading handwritten postal codes to sort mail at high speed. One of the intellectual leaders of that formative period, Satoshi Watanabe, proposed the remarkably evocative […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-1115","post","type-post","status-publish","format-standard","hentry","category-data-science"],"_links":{"self":[{"href":"https:\/\/www.mindware-jp.com\/en\/wp-json\/wp\/v2\/posts\/1115","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.mindware-jp.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.mindware-jp.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.mindware-jp.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.mindware-jp.com\/en\/wp-json\/wp\/v2\/comments?post=1115"}],"version-history":[{"count":2,"href":"https:\/\/www.mindware-jp.com\/en\/wp-json\/wp\/v2\/posts\/1115\/revisions"}],"predecessor-version":[{"id":1117,"href":"https:\/\/www.mindware-jp.com\/en\/wp-json\/wp\/v2\/posts\/1115\/revisions\/1117"}],"wp:attachment":[{"href":"https:\/\/www.mindware-jp.com\/en\/wp-json\/wp\/v2\/media?parent=1115"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.mindware-jp.com\/en\/wp-json\/wp\/v2\/categories?post=1115"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.mindware-jp.com\/en\/wp-json\/wp\/v2\/tags?post=1115"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}