April 2017 - intel-sgx-kernel-dev

by Jethro Beekman

In order to provide the ability to debug enclaves independently of their programming model, while supporting live attach (meaning the debugger might not have seen enclave creation), a debugger needs to following capabilities for a debugged process: 1) Identify enclaves a) Given a page, identify it as an enclave page b) Given an enclave page, find that enclave's base address c) Given a particular enclave, find the TCSes d) Given a particular enclave, get the attributes/miscselect 2) Read/write enclave memory 3) Find debugging symbols for enclaves I think (1)(a) and (1)(b) can be done using /proc/[pid]/maps, but I'd like confirmation. I think the following is true: If a page is in a memory range that is listed as mapped from the sgx device, it is an enclave page. From a memory range that is listed as mapped from the sgx device, you can compute the enclave base address as (start-offset). (2) is supported already by the driver using the ptrace interface. (1)(c) is necessary to find the SSAs and (1)(d) is necessary to determine SSAFRAMESIZE and make sense of the data in the SSAs. What would be the best way to expose this information? Best I can think of currently is a dedicated file in /proc/[pid] where the driver exposes this information which was originally provided at enclave creation time. I don't think (3) can universally be done without help from the user program. Yes, it might be possible to learn something from ELF headers but Linux does not generally have a requirement that user processes must have valid ELF information in their memory mappings. Most current userspace debug symbol loading is based on filenames. SGX does not provide any record of where enclave memory came from. Perhaps the create ioctl could include some user data that is later exposed again in the same proc file mentioned above. Frameworks and toolchains could establish a convention regarding this user data to find symbols. -- Jethro Beekman | Fortanix

4 years, 10 months

3
8
0 / 0

Remote attestation problem - Response headers

by cezane＠lsd.ufcg.edu.br

Hello. I do not know if this is the right list to ask about this problem. Sorry if it is not. I am doing the process of remote attestation in a nodejs application. When I try to do a post to IAS, I receive this error: error: { Error: Parse Error at TLSSocket.socketOnData (_http_client.js:411:20) at emitOne (events.js:96:13) at TLSSocket.emit (events.js:191:7) at readableAddChunk (_stream_readable.js:178:18) at TLSSocket.Readable.push (_stream_readable.js:136:10) at TLSWrap.onread (net.js:559:20) bytesParsed: 195, code: 'HPE_UNEXPECTED_CONTENT_LENGTH' } Searching about this problem, I found out that this is related to a malformed response from IAS, as stated in: -http://stackoverflow.com/questions/35525715/http-get-parse-error-code-hpe-unexpected-content-length -https://github.com/nodejs/node/issues/5754 -https://github.com/nodejs/node/issues/6517 Can anyone please explain me if this format of the response from IAS could be "fixed"? Thank you.

5 years

1
0
0 / 0

[PATCH] intel_sgx: compress sgx_encl_page flags and va_offset

by Sean Christopherson

Compress struct sgx_encl_page's flags and va_offset into a single 32-bit variable by assigning 20 bits of an unsigned int for flags and the remaining 12 bits for the VA offset. Signed-off-by: Sean Christopherson <sean.j.christopherson(a)intel.com> --- drivers/platform/x86/intel_sgx/sgx.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/platform/x86/intel_sgx/sgx.h b/drivers/platform/x86/intel_sgx/sgx.h index 82355bb..2b7a7f2 100644 --- a/drivers/platform/x86/intel_sgx/sgx.h +++ b/drivers/platform/x86/intel_sgx/sgx.h @@ -105,11 +105,11 @@ enum sgx_encl_page_flags { struct sgx_encl_page { unsigned long addr; - unsigned int flags; + unsigned int flags : 20, + va_offset : 12; struct sgx_epc_page *epc_page; struct list_head load_list; struct sgx_va_page *va_page; - unsigned int va_offset; }; struct sgx_tgid_ctx { -- 2.7.4

5 years, 1 month

3
4
0 / 0

[PATCH v2] intel_sgx: store va_offset in upper bits of page flags

by Sean Christopherson

Signed-off-by: Sean Christopherson <sean.j.christopherson(a)intel.com> --- drivers/platform/x86/intel_sgx/sgx.h | 17 ++++++++++++++++- drivers/platform/x86/intel_sgx/sgx_ioctl.c | 4 ++-- drivers/platform/x86/intel_sgx/sgx_page_cache.c | 2 +- drivers/platform/x86/intel_sgx/sgx_util.c | 2 +- 4 files changed, 20 insertions(+), 5 deletions(-) diff --git a/drivers/platform/x86/intel_sgx/sgx.h b/drivers/platform/x86/intel_sgx/sgx.h index e59ff01..430b5f2 100644 --- a/drivers/platform/x86/intel_sgx/sgx.h +++ b/drivers/platform/x86/intel_sgx/sgx.h @@ -101,6 +101,9 @@ static inline void sgx_free_va_slot(struct sgx_va_page *page, enum sgx_encl_page_flags { SGX_ENCL_PAGE_TCS = BIT(0), SGX_ENCL_PAGE_RESERVED = BIT(1), + + SGX_ENCL_PAGE_VA_OFFSET_SHIFT = 20, + SGX_ENCL_PAGE_VA_OFFSET_MASK = GENMASK(31, 20), }; struct sgx_encl_page { @@ -109,9 +112,21 @@ struct sgx_encl_page { struct sgx_epc_page *epc_page; struct list_head load_list; struct sgx_va_page *va_page; - unsigned int va_offset; }; +static inline unsigned int sgx_va_offset(struct sgx_encl_page *page) +{ + unsigned int shifted = (page->flags & SGX_ENCL_PAGE_VA_OFFSET_MASK); + return (shifted >> SGX_ENCL_PAGE_VA_OFFSET_SHIFT); +} + +static inline void sgx_set_va_offset(struct sgx_encl_page *page, + unsigned int va_offset) +{ + page->flags = (page->flags & ~SGX_ENCL_PAGE_VA_OFFSET_MASK) | + (va_offset << SGX_ENCL_PAGE_VA_OFFSET_SHIFT); +} + struct sgx_tgid_ctx { struct pid *tgid; struct kref refcount; diff --git a/drivers/platform/x86/intel_sgx/sgx_ioctl.c b/drivers/platform/x86/intel_sgx/sgx_ioctl.c index ba7c0d2..bca33f5 100644 --- a/drivers/platform/x86/intel_sgx/sgx_ioctl.c +++ b/drivers/platform/x86/intel_sgx/sgx_ioctl.c @@ -439,7 +439,7 @@ static int sgx_init_page(struct sgx_encl *encl, } entry->va_page = va_page; - entry->va_offset = va_offset; + sgx_set_va_offset(entry, va_offset); entry->addr = addr; return 0; @@ -753,7 +753,7 @@ static int __encl_add_page(struct sgx_encl *encl, if (ret) { kfree(req); sgx_free_va_slot(encl_page->va_page, - encl_page->va_offset); + sgx_va_offset(encl_page)); } mutex_unlock(&encl->lock); diff --git a/drivers/platform/x86/intel_sgx/sgx_page_cache.c b/drivers/platform/x86/intel_sgx/sgx_page_cache.c index 59a67cb..47a6221 100644 --- a/drivers/platform/x86/intel_sgx/sgx_page_cache.c +++ b/drivers/platform/x86/intel_sgx/sgx_page_cache.c @@ -290,7 +290,7 @@ static int __sgx_ewb(struct sgx_encl *encl, pginfo.linaddr = 0; pginfo.secs = 0; ret = __ewb(&pginfo, epc, - (void *)((unsigned long)va + encl_page->va_offset)); + (void *)((unsigned long)va + sgx_va_offset(encl_page))); kunmap_atomic((void *)(unsigned long)(pginfo.pcmd - pcmd_offset)); kunmap_atomic((void *)(unsigned long)pginfo.srcpge); diff --git a/drivers/platform/x86/intel_sgx/sgx_util.c b/drivers/platform/x86/intel_sgx/sgx_util.c index 1e9fa18..9cb7dd9 100644 --- a/drivers/platform/x86/intel_sgx/sgx_util.c +++ b/drivers/platform/x86/intel_sgx/sgx_util.c @@ -226,7 +226,7 @@ static int sgx_eldu(struct sgx_encl *encl, ret = __eldu((unsigned long)&pginfo, (unsigned long)epc_ptr, (unsigned long)va_ptr + - encl_page->va_offset); + sgx_va_offset(encl_page)); if (ret) { sgx_err(encl, "ELDU returned %d\n", ret); ret = -EFAULT; -- 2.7.4

5 years, 1 month

1
0
0 / 0

[PATCH v2 0/4] Combine EPC and VA page pointers into union

by Sean Christopherson

v2 of this set removes sgx_evict_page entirely instead of renaming sgx_ewb to sgx_evict_page. This patch set reduces the memory footprint of struct sgx_encl_page by combining its epc_page and va_page pointers into a union. Unionization of the EPC and VA pointers is made possible by allocating the VA slot at EWB and freeing it at ELDU, allowing sgx_encl_page to track the EPC page between EADD/ELDU and EWB, and the VA page between EWB and ELDU. Sean Christopherson (4): intel_sgx: track SECS eviction using its epc_page intel_sgx: delay VA slot allocation until EWB intel_sgx: combine epc_page and va_page into union intel_sgx: remove sgx_evict_page drivers/platform/x86/intel_sgx/sgx.h | 26 ++++++++----------- drivers/platform/x86/intel_sgx/sgx_ioctl.c | 105 ++++++++++++++++++++++++++++++++++++---------------------------------------- drivers/platform/x86/intel_sgx/sgx_page_cache.c | 64 ++++++++++++++++++++++++++++++++++------------ drivers/platform/x86/intel_sgx/sgx_util.c | 17 ++++++------- 4 files changed, 116 insertions(+), 96 deletions(-)

5 years, 1 month

1
4
0 / 0

[PATCH 0/4] Combine EPC and VA page pointers into union

by Sean Christopherson

This patch set reduces the memory footprint of struct sgx_encl_page by combining its epc_page and va_page pointers into a union. Unionization of the EPC and VA pointers is made possible by allocating the VA slot at EWB and freeing it at ELDU, allowing sgx_encl_page to track the EPC page between EADD/ELDU and EWB, and the VA page between EWB and ELDU. Sean Christopherson (4): intel_sgx: track SECS eviction using its epc_page intel_sgx: delay VA slot allocation until EWB intel_sgx: combine epc_page and va_page into union intel_sgx: rename EWB related functions drivers/platform/x86/intel_sgx/sgx.h | 26 +++++------- drivers/platform/x86/intel_sgx/sgx_ioctl.c | 105 +++++++++++++++++++++++-------------------------- drivers/platform/x86/intel_sgx/sgx_page_cache.c | 72 +++++++++++++++++++++++---------- drivers/platform/x86/intel_sgx/sgx_util.c | 17 ++++---- 4 files changed, 120 insertions(+), 100 deletions(-)

5 years, 1 month

3
11
0 / 0

[PATCH v3] intel_sgx: kill encl if vm_insert_pfn fails in fault

by Sean Christopherson

Update the EPC page tracking immediately after sgx_eldu and kill the encl if vm_insert_pfn fails. Previously we tried to EREMOVE the EPC page if vm_insert_pfn returned an error, but EREMOVE fails if there are active cpus in the enclave, in which case the driver would lose track of the EPC page. Signed-off-by: Sean Christopherson <sean.j.christopherson(a)intel.com> --- drivers/platform/x86/intel_sgx/sgx.h | 3 ++- drivers/platform/x86/intel_sgx/sgx_main.c | 2 +- drivers/platform/x86/intel_sgx/sgx_page_cache.c | 18 +++---------- drivers/platform/x86/intel_sgx/sgx_util.c | 36 ++++++++++++++++++++----- 4 files changed, 36 insertions(+), 23 deletions(-) diff --git a/drivers/platform/x86/intel_sgx/sgx.h b/drivers/platform/x86/intel_sgx/sgx.h index 82355bb..e59ff01 100644 --- a/drivers/platform/x86/intel_sgx/sgx.h +++ b/drivers/platform/x86/intel_sgx/sgx.h @@ -198,7 +198,8 @@ int sgx_eremove(struct sgx_epc_page *epc_page); struct vm_area_struct *sgx_find_vma(struct sgx_encl *encl, unsigned long addr); void sgx_zap_tcs_ptes(struct sgx_encl *encl, struct vm_area_struct *vma); -void sgx_invalidate(struct sgx_encl *encl); +void sgx_invalidate(struct sgx_encl *encl, bool flush_cpus); +void sgx_flush_cpus(struct sgx_encl *encl); int sgx_find_encl(struct mm_struct *mm, unsigned long addr, struct vm_area_struct **vma); diff --git a/drivers/platform/x86/intel_sgx/sgx_main.c b/drivers/platform/x86/intel_sgx/sgx_main.c index b2a1bc4..5890643 100644 --- a/drivers/platform/x86/intel_sgx/sgx_main.c +++ b/drivers/platform/x86/intel_sgx/sgx_main.c @@ -243,7 +243,7 @@ static int sgx_pm_suspend(struct device *dev) list_for_each_entry(ctx, &sgx_tgid_ctx_list, list) { list_for_each_entry(encl, &ctx->encl_list, encl_list) { - sgx_invalidate(encl); + sgx_invalidate(encl, false); encl->flags |= SGX_ENCL_SUSPEND; flush_work(&encl->add_page_work); } diff --git a/drivers/platform/x86/intel_sgx/sgx_page_cache.c b/drivers/platform/x86/intel_sgx/sgx_page_cache.c index 4ffde27..59a67cb 100644 --- a/drivers/platform/x86/intel_sgx/sgx_page_cache.c +++ b/drivers/platform/x86/intel_sgx/sgx_page_cache.c @@ -221,15 +221,6 @@ static void sgx_isolate_pages(struct sgx_encl *encl, mutex_unlock(&encl->lock); } -static void sgx_ipi_cb(void *info) -{ -} - -static void sgx_flush_cpus(struct sgx_encl *encl) -{ - on_each_cpu_mask(mm_cpumask(encl->mm), sgx_ipi_cb, NULL, 1); -} - static void sgx_eblock(struct sgx_encl *encl, struct sgx_epc_page *epc_page) { @@ -242,8 +233,7 @@ static void sgx_eblock(struct sgx_encl *encl, if (ret) { sgx_crit(encl, "EBLOCK returned %d\n", ret); - sgx_invalidate(encl); - sgx_flush_cpus(encl); + sgx_invalidate(encl, true); } } @@ -259,8 +249,7 @@ static void sgx_etrack(struct sgx_encl *encl) if (ret) { sgx_crit(encl, "ETRACK returned %d\n", ret); - sgx_invalidate(encl); - sgx_flush_cpus(encl); + sgx_invalidate(encl, true); } } @@ -327,8 +316,7 @@ static bool sgx_ewb(struct sgx_encl *encl, if (ret) { /* make enclave inaccessible */ - sgx_invalidate(encl); - sgx_flush_cpus(encl); + sgx_invalidate(encl, true); if (ret > 0) sgx_err(encl, "EWB returned %d, enclave killed\n", ret); return false; diff --git a/drivers/platform/x86/intel_sgx/sgx_util.c b/drivers/platform/x86/intel_sgx/sgx_util.c index 25e949d..1e9fa18 100644 --- a/drivers/platform/x86/intel_sgx/sgx_util.c +++ b/drivers/platform/x86/intel_sgx/sgx_util.c @@ -120,7 +120,7 @@ void sgx_zap_tcs_ptes(struct sgx_encl *encl, struct vm_area_struct *vma) } } -void sgx_invalidate(struct sgx_encl *encl) +void sgx_invalidate(struct sgx_encl *encl, bool flush_cpus) { struct vm_area_struct *vma; unsigned long addr; @@ -135,6 +135,18 @@ void sgx_invalidate(struct sgx_encl *encl) } encl->flags |= SGX_ENCL_DEAD; + + if (flush_cpus) + sgx_flush_cpus(encl); +} + +static void sgx_ipi_cb(void *info) +{ +} + +void sgx_flush_cpus(struct sgx_encl *encl) +{ + on_each_cpu_mask(mm_cpumask(encl->mm), sgx_ipi_cb, NULL, 1); } /** @@ -315,10 +327,12 @@ static struct sgx_encl_page *sgx_do_fault(struct vm_area_struct *vma, if (rc) goto out; - rc = vm_insert_pfn(vma, entry->addr, PFN_DOWN(epc_page->pa)); - if (rc) - goto out; - + /* Track the EPC page even if vm_insert_pfn fails; we need to ensure + * the EPC page is properly freed and we can't do EREMOVE right away + * because EREMOVE may fail due to an active cpu in the enclave. We + * can't call vm_insert_pfn before sgx_eldu because SKL signals #GP + * instead of #PF if the EPC page is invalid. + */ encl->secs_child_cnt++; entry->epc_page = epc_page; @@ -328,9 +342,19 @@ static struct sgx_encl_page *sgx_do_fault(struct vm_area_struct *vma, /* Do not free */ epc_page = NULL; + list_add_tail(&entry->load_list, &encl->load_list); + + rc = vm_insert_pfn(vma, entry->addr, PFN_DOWN(entry->epc_page->pa)); + if (rc) { + /* Kill the enclave if vm_insert_pfn fails; failure only occurs + * if there is a driver bug or an unrecoverable issue, e.g. OOM. + */ + sgx_crit(encl, "vm_insert_pfn returned %d\n", rc); + sgx_invalidate(encl, true); + goto out; + } sgx_test_and_clear_young(entry, encl); - list_add_tail(&entry->load_list, &encl->load_list); out: mutex_unlock(&encl->lock); if (epc_page) -- 2.7.4

5 years, 1 month

2
2
0 / 0

[PATCH] intel_sgx: use offset bits of EPC page to store its bank

by Sean Christopherson

Store the bank index of an EPC page in the unused page offset of the EPC address. This allows sgx_get_epc_page to calculate the virtual address of an EPC page in constant time (for x64-86) and without any branches. Rename sgx_epc_page's pa to addr to reflect that it tracks more than just the PA and add macros to get the PA and BANK. Signed-off-by: Sean Christopherson <sean.j.christopherson(a)intel.com> --- arch/x86/include/asm/sgx.h | 7 +++++-- drivers/platform/x86/intel_sgx/sgx.h | 2 +- drivers/platform/x86/intel_sgx/sgx_ioctl.c | 2 +- drivers/platform/x86/intel_sgx/sgx_main.c | 2 +- drivers/platform/x86/intel_sgx/sgx_page_cache.c | 19 +++++-------------- drivers/platform/x86/intel_sgx/sgx_util.c | 2 +- 6 files changed, 14 insertions(+), 20 deletions(-) diff --git a/arch/x86/include/asm/sgx.h b/arch/x86/include/asm/sgx.h index 233d3a6..d8706f6 100644 --- a/arch/x86/include/asm/sgx.h +++ b/arch/x86/include/asm/sgx.h @@ -342,10 +342,13 @@ static inline int __emodt(struct sgx_secinfo *secinfo, void *epc) struct sgx_encl; struct sgx_epc_page { - resource_size_t pa; - struct list_head free_list; + resource_size_t addr; + struct list_head free_list; }; +#define EPC_PA(entry) ((entry)->addr & PAGE_MASK) +#define EPC_BANK(entry) ((entry)->addr & ~PAGE_MASK) + extern struct sgx_epc_page *sgx_alloc_page(unsigned int flags); extern int sgx_free_page(struct sgx_epc_page *entry, struct sgx_encl *encl); extern void *sgx_get_page(struct sgx_epc_page *entry); diff --git a/drivers/platform/x86/intel_sgx/sgx.h b/drivers/platform/x86/intel_sgx/sgx.h index 82355bb..8c0e19a 100644 --- a/drivers/platform/x86/intel_sgx/sgx.h +++ b/drivers/platform/x86/intel_sgx/sgx.h @@ -222,7 +222,7 @@ enum sgx_alloc_flags { }; int ksgxswapd(void *p); -int sgx_page_cache_init(resource_size_t start, unsigned long size); +int sgx_page_cache_init(resource_size_t start, unsigned long size, int epc_bank); void sgx_page_cache_teardown(void); struct sgx_epc_page *sgx_alloc_page(unsigned int flags); int sgx_free_page(struct sgx_epc_page *entry, struct sgx_encl *encl); diff --git a/drivers/platform/x86/intel_sgx/sgx_ioctl.c b/drivers/platform/x86/intel_sgx/sgx_ioctl.c index ba7c0d2..cb5cfe1 100644 --- a/drivers/platform/x86/intel_sgx/sgx_ioctl.c +++ b/drivers/platform/x86/intel_sgx/sgx_ioctl.c @@ -244,7 +244,7 @@ static bool sgx_process_add_page_req(struct sgx_add_page_req *req) goto out; } - ret = vm_insert_pfn(vma, encl_page->addr, PFN_DOWN(epc_page->pa)); + ret = vm_insert_pfn(vma, encl_page->addr, PFN_DOWN(EPC_PA(epc_page))); if (ret) goto out; diff --git a/drivers/platform/x86/intel_sgx/sgx_main.c b/drivers/platform/x86/intel_sgx/sgx_main.c index b2a1bc4..84db81d 100644 --- a/drivers/platform/x86/intel_sgx/sgx_main.c +++ b/drivers/platform/x86/intel_sgx/sgx_main.c @@ -290,7 +290,7 @@ static int sgx_dev_init(struct device *dev) } #endif ret = sgx_page_cache_init(sgx_epc_banks[i].start, - sgx_epc_banks[i].end - sgx_epc_banks[i].start); + sgx_epc_banks[i].end - sgx_epc_banks[i].start, i); if (ret) { sgx_nr_epc_banks = i+1; goto out_iounmap; diff --git a/drivers/platform/x86/intel_sgx/sgx_page_cache.c b/drivers/platform/x86/intel_sgx/sgx_page_cache.c index 4ffde27..44a39ae 100644 --- a/drivers/platform/x86/intel_sgx/sgx_page_cache.c +++ b/drivers/platform/x86/intel_sgx/sgx_page_cache.c @@ -428,7 +428,7 @@ int ksgxswapd(void *p) return 0; } -int sgx_page_cache_init(resource_size_t start, unsigned long size) +int sgx_page_cache_init(resource_size_t start, unsigned long size, int epc_bank) { unsigned long i; struct sgx_epc_page *new_epc_page, *entry; @@ -438,7 +438,7 @@ int sgx_page_cache_init(resource_size_t start, unsigned long size) new_epc_page = kzalloc(sizeof(*new_epc_page), GFP_KERNEL); if (!new_epc_page) goto err_freelist; - new_epc_page->pa = start + i; + new_epc_page->addr = (start + i) | epc_bank; spin_lock(&sgx_free_list_lock); list_add_tail(&new_epc_page->free_list, &sgx_free_list); @@ -584,19 +584,10 @@ EXPORT_SYMBOL(sgx_free_page); void *sgx_get_page(struct sgx_epc_page *entry) { #ifdef CONFIG_X86_32 - return kmap_atomic_pfn(PFN_DOWN(entry->pa)); + return kmap_atomic_pfn(PFN_DOWN(EPC_PA(entry))); #else - int i; - - for (i = 0; i < sgx_nr_epc_banks; i++) { - if (entry->pa < sgx_epc_banks[i].end && - entry->pa >= sgx_epc_banks[i].start) { - return sgx_epc_banks[i].mem + - (entry->pa - sgx_epc_banks[i].start); - } - } - - return NULL; + int i = EPC_BANK(entry); + return sgx_epc_banks[i].mem + (EPC_PA(entry) - sgx_epc_banks[i].start); #endif } EXPORT_SYMBOL(sgx_get_page); diff --git a/drivers/platform/x86/intel_sgx/sgx_util.c b/drivers/platform/x86/intel_sgx/sgx_util.c index 25e949d..315f4ea 100644 --- a/drivers/platform/x86/intel_sgx/sgx_util.c +++ b/drivers/platform/x86/intel_sgx/sgx_util.c @@ -315,7 +315,7 @@ static struct sgx_encl_page *sgx_do_fault(struct vm_area_struct *vma, if (rc) goto out; - rc = vm_insert_pfn(vma, entry->addr, PFN_DOWN(epc_page->pa)); + rc = vm_insert_pfn(vma, entry->addr, PFN_DOWN(EPC_PA(epc_page))); if (rc) goto out; -- 2.7.4

5 years, 1 month

2
1
0 / 0

[PATCH 0/7] 'Unified EPC' changes to support KVM SGX

by Kai Huang

With KVM SGX virtualization support, both SGX driver and KVM needs to use EPC. As we have agreed we need to use 'unified model' for EPC management, in which EPC is managed by SGX driver and KVM just calls SGX driver's interfaces for EPC allocation/free, etc. This patch series modify SGX driver to suit KVM SGX virtualization. Patch 1 moves ENCLS error codes from uapi/asm/sgx.h to asm/sgx.h. Patch 2 refined muitiple EPC banks handling. I sent out this patch separately several days ago but I also included it here (with minor modification) as further patches depend on it. Patch 3-5 exposes EPC management interfaces for KVM, with some changes applied to suit KVM. Patch 6 is just renaming and is trival. I did it because KVM uses 'list' instead of 'free_list' and I personally think 'list' is more reasonable. Patch 7 is not related to 'unified EPC' and you can ignore it if you have already done it in other branches in your code. This series are just changes for 'unified EPC'. We don't support evicting EPC from KVM guests (unified oversubscription) yet. I'll send further patches for 'unified oversubscription' once they are done. Btw I have tested this series on KVM side, but I didn't manage to test with any testsuite for host SGX application because neither SGXBat nor SgxDriverValidation worked on my machine (SgxDriverValidation worked in KVM guest where SDK's SGX driver was used, but in host I cannot use SDK's SGX driver but can only use upstreaming driver, so SgxDriverValidation cannot work). It will be apprecaited if you can do some sanity test on host SGX app side. Kai Huang (7): intel_sgx: move ENCLS error codes from uapi/sgx.h to sgx.h intel_sgx: refine page cache initialization for multiple EPC banks intel_sgx: expose sgx_pfn_to_epc_page intel_sgx: expose sgx_kmap{kunmap}_epc_page intel_sgx: expose sgx_alloc{free}_page intel_sgx: rename free_list to list for 'struct sgx_epc_page' intel_sgx: add TRIM EPC page type definition. arch/x86/include/asm/sgx.h | 47 ++++++++ arch/x86/include/uapi/asm/sgx.h | 26 +---- drivers/platform/x86/intel_sgx.h | 26 +---- drivers/platform/x86/intel_sgx_ioctl.c | 54 ++++++---- drivers/platform/x86/intel_sgx_main.c | 40 ++----- drivers/platform/x86/intel_sgx_page_cache.c | 162 +++++++++++++++++++--------- drivers/platform/x86/intel_sgx_util.c | 133 ++++++++++++++++++----- drivers/platform/x86/intel_sgx_vma.c | 12 +-- 8 files changed, 316 insertions(+), 184 deletions(-) -- 2.9.3

5 years, 1 month

3
42
0 / 0

[PATCH v2] intel_sgx: correctly handle vm_insert_pfn failure

by Sean Christopherson

Update the EPC page tracking immediately after sgx_eldu, and retry vm_insert_pfn on a future fault if vm_insert_pfn fails. Previously we tried to EREMOVE the EPC page if vm_insert_pfn return an error, but EREMOVE fails if there are active cpus in the enclave, in which case the driver would effectively lose track of the EPC page. Signed-off-by: Sean Christopherson <sean.j.christopherson(a)intel.com> --- drivers/platform/x86/intel_sgx.h | 1 + drivers/platform/x86/intel_sgx_util.c | 41 +++++++++++++++++++++++++++-------- 2 files changed, 33 insertions(+), 9 deletions(-) diff --git a/drivers/platform/x86/intel_sgx.h b/drivers/platform/x86/intel_sgx.h index adb5b17..8b14b1f 100644 --- a/drivers/platform/x86/intel_sgx.h +++ b/drivers/platform/x86/intel_sgx.h @@ -106,6 +106,7 @@ static inline void sgx_free_va_slot(struct sgx_va_page *page, enum sgx_encl_page_flags { SGX_ENCL_PAGE_TCS = BIT(0), SGX_ENCL_PAGE_RESERVED = BIT(1), + SGX_ENCL_PAGE_PTE_VALID = BIT(2), }; struct sgx_encl_page { diff --git a/drivers/platform/x86/intel_sgx_util.c b/drivers/platform/x86/intel_sgx_util.c index 234a5fb..f4bbd38 100644 --- a/drivers/platform/x86/intel_sgx_util.c +++ b/drivers/platform/x86/intel_sgx_util.c @@ -289,6 +289,22 @@ static int sgx_eldu(struct sgx_encl *encl, return ret; } +static inline int sgx_fault_insert_pfn(struct vm_area_struct *vma, + struct sgx_encl *encl, + struct sgx_encl_page *entry) +{ + int ret = 0; + if (!(entry->flags & SGX_ENCL_PAGE_PTE_VALID)) { + ret = vm_insert_pfn(vma, entry->addr, PFN_DOWN(entry->epc_page->pa)); + if (!ret) { + entry->flags |= SGX_ENCL_PAGE_PTE_VALID; + sgx_test_and_clear_young(entry, encl); + } + } + + return ret; +} + static struct sgx_encl_page *sgx_do_fault(struct vm_area_struct *vma, unsigned long addr, unsigned int flags) { @@ -340,6 +356,9 @@ static struct sgx_encl_page *sgx_do_fault(struct vm_area_struct *vma, /* Legal race condition, page is already faulted. */ if (entry->epc_page) { + rc = sgx_fault_insert_pfn(vma, encl, entry); + if (rc) + goto out; if (reserve) entry->flags |= SGX_ENCL_PAGE_RESERVED; goto out; @@ -369,22 +388,26 @@ static struct sgx_encl_page *sgx_do_fault(struct vm_area_struct *vma, if (rc) goto out; - rc = vm_insert_pfn(vma, entry->addr, PFN_DOWN(epc_page->pa)); - if (rc) - goto out; - + /* Track the page, even if vm_insert_pfn fails. We can't EREMOVE + * the page because EREMOVE may fail due to an active cpu in the + * enclave. We can't call vm_insert_pfn before sgx_eldu because + * SKL platforms signal #GP instead of #PF if the EPC page is invalid. + */ encl->secs_child_cnt++; entry->epc_page = epc_page; - - if (reserve) - entry->flags |= SGX_ENCL_PAGE_RESERVED; + entry->flags &= ~SGX_ENCL_PAGE_PTE_VALID; /* Do not free */ epc_page = NULL; - - sgx_test_and_clear_young(entry, encl); list_add_tail(&entry->load_list, &encl->load_list); + + rc = sgx_fault_insert_pfn(vma, encl, entry); + if (rc) + goto out; + + if (reserve) + entry->flags |= SGX_ENCL_PAGE_RESERVED; out: mutex_unlock(&encl->lock); if (epc_page) -- 2.7.4

5 years, 1 month

3
6
0 / 0

2022

2021

2020

2019

2018

2017

2016

intel-sgx-kernel-dev April 2017