I'll try to explain my point of view on this to the extent that I myself understand it.

First to me what you call "personhood", which you define as that which makes it wrong to kill (or I would assume as well cause suffering to) another human being, is based on the capacity of the being in question to experience suffering. I think that humans have a much higher capacity for suffering than most (if not all) other terrestrial animals due to their more complex nervous systems. Part of this increased capacity may be due to increased cognitive capacity per se (for example I will suffer if I think my life is in danger whereas an animal may not be able to conceptualize that danger and hence will not experience suffering in the same situation). Humans clearly have greater sapience than animals and this can lead to greater suffering. It is also possible that humans have a greater intrinsic capacity for subjective experience independently of any cognitive process. If this is true I would say that humans have greater sentience than animals. I don't know if it's true because I lack a conceptual model for understanding the nature of sentience and what physical processes may give rise to it.

In the absence of such a model, I can only guess and my guess is that sentience somehow arises from the interaction between information and matter.

I think it's likely that deterministic computers will achieve human level cognitive abilities in the relatively near future. I know of no cognitive function that humans perform which I can't imagine being executed on a sufficiently powerful computer. I would consider such machines to be sapient but as far as I can see they would have no sentience whatsoever (not even as much as lower animals).

As for your last question:

It seems to me that human beings are constantly falling into the same intellectual trap. That trap consists of believing that their scientific theories are not only accurate but also complete. At the end of the 19th century physicists believed they knew essentially everything there was to know about physics (except for a couple of pesky anomalies that could easily be swept under the rug). Ten years ago it was common wisdom that the majority of human DNA was junk which served no purpose. Only now are we beginning to understand the role that "junk" plays in regulating gene expression.

I think that current scientific theories are not capable of explaining the nature of sentience or of predicting which physical systems will be sentient and which not. I'm not sure if that makes me a supernaturalist or not.

How would you tell the difference between a human-level AI and a human? Sentience is not a well defined term. The best thing you can do is to ask the intelligence in question whether has subjective experience, if it says yes you have to believe that. I couldn't prove to you that I have subjective experiences. I think it's irrational to say that every cognitive ability a human has can be simulated perfectly fine on a computer, but only a biological brain is capable of subjective experiences.

Thanks for the detailed response.

>It is also possible that humans have a greater intrinsic capacity for subjective experience independently of any cognitive process.

I must disagree with you here - I would say that subjective experience is a cognitive process.

Of course, this is a speculative opinion; but it seems likely to me:

We know that we can have simple algorithmic and statistical processes on neural nets; we know we can build very complex processes from simple building blocks; we know the brain is full of neural nets; we know that the high level cognitive processes are build on the neural nets; and that if we interfere with the brain, we disturb the personhood.

To me, this seems like a pretty good case for applying occams razor and deciding that the personhood comes from the (big, incredibly complex) neural nets.

Loosely, I believe that if we could write a sufficiently powerful piece of software, and if we constructed it so that that was capable of complex introspection, then it would say it had a subjective experience. And I would tend to believe it. Such a feat is clearly some distance from the current state of the art and we have no idea how to do this - but still.

>I think that current scientific theories are not capable of explaining the nature of sentience or of predicting which physical systems will be sentient and which not. I'm not sure if that makes me a supernaturalist or not.

This is tricky territory.

Its reasonable to believe our scientific theories are incomplete. Certainly, we have a long way to go, scientifically, before we understand how minds operate.

But, lets look at our physics knowledge. Our physics models seem to be pretty good. Unless we are at very high energies, they seem to do a very accurate job of describing the world. We've built components with that knowledge, that have been reliable; our abstractions do not seem very leaky.

It is important to realise, that while relativity did come along and wipe away aspects of the newtonian worldview, newtonian physics is still accurate for deciding where to throw the ball to get it into the net. The more detailed physics was only relevant when working with certain situations that demanded the lower level of abstraction.

I don't believe nature evolved brains that require subtle properties of high energy physics to work. I'd say the systems underlying them aren't even quantum in nature. Some scientists would dispute this - notably Roger Penrose.

But is evolution really going to build brains, and processing software - which are present, in some form, in a vast range of species - out of some mysterious quantum - or lower level - physics?

Evolution tends to be about bootstrapping. Higher level systems are always assembled out of lower level building blocks. Thats why I have cells, tissues, organs - motifs and structure that repeat themselves.

If you evolve a system that is sensitive to the smallest changes in the properties of its smallest units (quantum interactions, say) its much hard, in terms of the search required. It also is going to be much more fragile.

Even neurons are a pretty high level construct, from a physics point of view; high enough so that our current physics probably models their functional characteristics completely. (Although we have yet to fully apply our current physics to analyse them completely; ANNs as we see them in AI are but the crudest approximations, and shouldnt be considered descriptive.).

So I would tend to believe that we don't need any new physics to understand how minds work.

If you are saying that there is some strange new physics needed to describe how the information relates to matter, then I think you are, functionally, a supernaturalist. I also put Penrose in that box, though, so its a personal opinion, and the jury is still out.

If you think that there is a lot of science yet to be discovered, to come up with theories of how computational building blocks are made from neural architecture, and what the 'algorithms' that run on that architecture are, then I agree with you.

I am glad that we can have this discussion, despite its difficulty. It is an intensely interesting subject, and I can't resist.

For the poster below, thank you for bringing up the p-zombie argument. Indeed, with links to deconstructionism and physicalism, how can you even tell that another human is "human"? Can you prove that that individual is conscious, has subjective experience, and has a soul? Or is it, in fact, an assumption that all humans possess these things? This fundamental assumption is put to the test if a computer can perfectly simulate a human mind. Perhaps we will see that such notions are purely subjective constructs, like "cold". Though we may feel "cold", it is a sensation and an experience. That experience is not provable, even if we might get everyone to agree that -50C is "cold." All we can prove is the behavior of the brain, and assume that that behavior is linked to experience in general.